Image conversion device image conversion method and image projection device

ABSTRACT

An image processing apparatus for reducing a load of processing of an interpolation operation at the time of a correction of keystone distorted image is provided. The image processing apparatus has an interpolating unit ( 11, 12 ) for receiving as input original image data, executing a first interpolation operation a plurality of times with respect to a plurality of original pixel data arranged in one direction of a vertical direction or a horizontal direction in the original image data, executing a second interpolation operation in the other direction, and generating new image data and a storing unit ( 13 ) for storing the interpolated data (Va to Vd) obtained by the first interpolation operations, wherein, where a combination of a plurality of original pixel data is the same as a combination used when the interpolated data (Va= to Vd=) already stored in the storing unit ( 13 ) were calculated, the interpolating unit ( 12 ) reads out that interpolated data from the storing unit ( 13 ) and uses the same for the second interpolation operation.

TECHNICAL FIELD

The present invention relates to an image conversion apparatus and animage conversion method for converting an input original image to a newimage by an interpolation operation and to an image projection apparatusfor generating data of an image having distortion corrected on asubstantially vertical projection surface by an interpolation operation.

BACKGROUND ART

An image projection apparatus referred to as a “projector” has a displaymeans, for example, an LCD (liquid crystal display). The display meansis made to display the image and project the image onto a projectionsurface such as an external screen. At this time, if a projection angleof the image from the liquid crystal projector with respect to theprojection surface is inclined, the image which should inherently berectangular becomes distorted in a trapezoidal state on the screen.

For this reason, a liquid crystal projector provided with a so-calledkeystone correction function for reversely distorting the image on theliquid crystal panel to correct the trapezoidal distortion of the imageon the screen has been known.

Usually, a projector is provided with a pixel number conversion functionin accordance with the type or resolution of various input images or forrealizing a PinP (Picture in Picture) function. As the function forpixel number conversion of a projector, there are a function forgenerating addresses after conversion and an interpolation operationfunction for generating one pixel data from among a plurality of pixeldata selected from an input image by an interpolation operation forevery generated address.

FIG. 28A-1, FIG. 28A-2, FIG. 28B-1, and FIG. 28B-2 show an originalimage, an image after conversion, and images of the address conversionusing trapezoidal distortion correction in a horizontal direction as anexample.

In the image conversion, generally, as shown in FIG. 28B-1, addresses(pixel position data) of an image having the same distortion as thedistortion occurring on the projection surface are generated inaccordance with the projection position. Below, addresses of pixels atwhich such data is to be generated by interpolation will be referred toas “interpolation addresses”, and the new data will be referred to as“interpolated pixel data”.

In order to generate interpolated pixel data corresponding to theinterpolation addresses by the interpolation operation, for example, foreach address, a plurality of peripheral original pixel data around thecorresponding location of the original image are selected by apredetermined rule and added weighted by a predetermined rule. The datais generated repeatedly to generate all interpolated pixel data, then,as shown in FIG. 28B-2, the group of the generated interpolated pixeldata is converted in addresses all together.

In trapezoidal distortion correction in the horizontal direction, theinterpolation addresses are not given in parallel with respect to thescanning line direction as in FIG. 28B-1, but are given while beinginclined to a certain slant for every line. The intervals between thecenters of the interpolation pixels serving as sampling points at thistime are not constant and change nonlinearly. Further, the same can besaid for lines as well. The intervals between lines are also notconstant. When giving interpolation addresses obliquely in this way,generating interpolated pixel data, and allocating them to theinterpolation addresses, the result is that an image intentionallydistorted reverse to the image on the projection surface as indicated bythe image in FIG. 28A-2 is obtained. If projecting this intentionallydistorted image onto the projection surface, a rectangular image havingdistortion cancelled out is obtained.

While correction of distortion in the horizontal direction was explainedabove, distortion in the vertical direction can also be corrected by thesame method. At the time of correction of distortion in the verticaldirection, a group of addresses representing the trapezoidal distortedimage flaring out upward or downward is generated, and an interpolationoperation is carried out at each of the address points.

On the other hand, in the case of projection from a slant having anyangles horizontally and vertically with respect to the projectionsurface, the distortions in the horizontal direction and the verticaldirection are combined and the image becomes complicatedly distorted, sogeneration of a group of addresses representing the distortion iscomplicated. Note if it were only possible to efficiently generate thegroup of addresses, the interpolation operation itself could be carriedout in the same way as the other case mentioned above.

When including a distortion component in the horizontal direction asmentioned above, as shown in FIG. 28B-1, a line connecting theinterpolation addresses will obliquely cross a plurality of horizontallines (a plurality of horizontal scanning lines) of the original image.For this reason, in the interpolation of a pixel, it is necessary to usethe plurality of pixel data of the original image existing around theinterpolation address point horizontally and vertically, so atwo-dimensional interpolation operation becomes necessary. For thisoperation, a two-dimensional interpolation filter can be used. Note,generally, use is made of two one-dimensional interpolation filtersindependent vertically and horizontally for reasons of operation scale,restriction of the bit size of the memory, degree of freedom of setup,etc. For example, by passing the original image data through thevertical one-dimensional interpolation filter first and then furtherpassing the interpolated data generated by this through the horizontalone-dimensional interpolation filter, new pixel data is generated.

FIG. 29 shows an example of a one-dimensional interpolation operationfor generating one pixel data by a convolution operation by a four-tapfilter.

An interpolation coefficient is determined by the distance between theinterpolation position and the pixel data, therefore an interpolationcoefficient (filter coefficient) with respect to each phase can berepresented by a function h(x) of the distance in an x-axis direction atthis time. Accordingly, pixel data g of the illustrated interpolationpoint Q at this time can be represented by the convolution operationshown in equation (1) using original pixel data A, B, C, and D.q=Axh(−6/5)+Bxh(−1/5)+Cxh(4/5)+Dxh(9/5)  (1)

In actuality, a variety of values can be considered for theinterpolation function h(x). Also, the image quality can be changed bychanging the characteristics of the interpolation filter.

FIG. 30 shows the configuration of a filter unit particularly performinga one-dimensional filter operation independently two times in an imageconversion block.

A filter unit 200 has a vertical interpolation filter (VIF) 201, ahorizontal interpolation filter (HIF) 202, a selector (SEL) 203, and astoring means 204. The original pixel data and a set of filtercoefficients are input to the vertical interpolation filter 201 first.By passing the original pixel data through the vertical interpolationfilter 201, the filter operation shown in equation (1) is executed andinterpolated data generated by using for example four vertical directionpixel data is output. This is repeated at for example four columns ofpixels around the interpolation point, whereby four vertical directioninterpolated data Va, Vb, Vc, and Vd are sequentially output from thevertical interpolation filter 201 to the selector 203. The selector 203sequentially distributes the four vertical direction interpolated dataVa, Vb, Vc, and Vd to predetermined storage portions (or predeterminedaddresses) in the storing means 204 while switching the output. When thefour interpolated data Va, Vb, Vc, and Vd are assembled, the storingmeans 204 outputs them to the horizontal interpolation filter 202. Thehorizontal interpolation filter 202 executes a one-dimensional(horizontal direction) filter operation in accordance with the input setof filter coefficients and outputs the result as new pixel data.

In this way, in order to generate one new pixel data, horizontal andvertical (N×M) number of data of the original image, i.e., 4×4=16 datain the above example, are used. In order to generate one new pixel data,the interpolation operation by the N-tap vertical interpolation filteris carried out M number of times (N=M=4 in the above example). For the Mnumber of one-dimensional interpolated data V1, V2, . . . , VM generatedby this, an interpolation operation by the M-tap horizontalinterpolation filter is executed only one time.

In the interpolation operation processing of a pixel carried out in theimage conversion unit of the image projection apparatus mentioned above,the number of the original pixel data extracted is set to N in thevertical direction and M in the horizontal direction (N and M: naturalnumbers of 2 or more). In general, the higher the order of theinterpolation function h(x) of equation (1) used for realizing highprecision interpolation, the larger the numbers N and M of this originalpixel data extracted and the larger the number of taps of the filtertoo. Further, in order to perform the distortion correction in thehorizontal direction with a high precision, a certain number of originalpixel data becomes necessary in not only the horizontal direction, butalso the vertical direction. In this case, in order to raise the degreeof freedom of placement of the projector, it is important that largedistortion of an image can be corrected. From this viewpoint, thenumbers N and M of the original pixel data extracted are set large inadvance.

In the filter unit 200 having the configuration shown in FIG. 30, whennew pixel data is generated, if the interpolated data Va, Vb, Vc, and Vdoutput from the vertical interpolation filter 201 are input to thestoring means 204, the interpolated data Va=, Vb=, Vc=, and Vd= used forthe generation of the pixel data before that are rewritten. With thefilter unit 200 of this configuration, M (four in the present example)number of interpolated data are generated for every generation of newimage data.

In the filter unit 200 illustrated in FIG. 30, however, some of theinterpolated data are often the same at the time of generation ofcontinuous image data. In this case, wasteful processing for generatingdata the same as already generated and stored data and rewriting thestored content in the storing means 204 by this is carried out. In orderto improve the precision of the interpolation processing or the degreeof freedom of the arrangement of the projector, when the numbers N and Mof the original pixel data extracted are large, the same operation isoften repeated in the vertical interpolation filter 201. Further, thefrequency of repetition of operations of the same content in thisvertical interpolation increases as the vertical direction distortioncomponent becomes dominant in comparison with the horizontal directiondistortion component. Further, even when there are few wasted repeatedoperations, the generation of addresses of the oblique distortioncorrection was complex, so the processing time sometimes increased.

In the image projection apparatus mentioned above, wasted repetition ofthe same operations frequently occurred due to the above reason or theimprovement of the overall operation speed was unnecessarily limited dueto the complicated address computations. For this reason, variousrestrictions arose, for example, the clock frequency etc. of theportions communicating with the memory had to be raised or the bit sizeof the memory had to be increased.

DISCLOSURE OF THE INVENTION

A first object of the present invention is to provide an imageconversion apparatus and an image conversion method improving aprocessing speed of an interpolation operation and reducing a load on arequired memory.

A second object of the present invention is to provide an imageprojection apparatus for generating data of an image able to correctdistortion on a projection surface by using a method of an interpolationoperation able to improve the processing speed of the interpolationoperation and reduce the load on the memory.

According to a first aspect of the present invention, there is providedan image conversion apparatus comprising an interpolating means forexecuting a first interpolation operation by a plurality of originalpixel data arranged in one direction of either a vertical or horizontaldirection of the input original image, executing a second interpolationoperation in the other direction different from the one direction byusing a plurality of interpolated data obtained by the firstinterpolation operation, and generating new image data at aninterpolation point, and a storing means for storing the interpolateddata obtained by the first interpolation operation, wherein theinterpolating means reads out the interpolated data from the storingmeans if a combination of the plurality of original pixel data is thesame as a combination used when calculating the interpolated dataalready stored in the storing means and uses the same for the secondinterpolation operation.

According to a second aspect of the present invention, there is providedan image projection apparatus having a display means having displaypixels arranged in a matrix in first and second directions orthogonal toeach other and a projecting means for projecting an image displayed onthe display means to a projection surface by utilizing light from alight source and having a function of converting an input original imageto an image having distortion corrected in accordance with an angle ofthe projection with respect to a normal line of the projection surfacewhen projecting the image to the projection surface, comprising anaddress generating means for generating addresses of an image sufferingfrom distortion linked with positions of the display on the displaymeans; a mapping means for linking pixel positions of the original imagewithout distortion with the addresses of the image suffering fromdistortion; a selecting means for selecting a plurality of originalimage data in the second direction for every intersecting point based onintersecting points between an address line of the image suffering fromdistortion generated by the address generating means corresponding tothe displayed pixels arranged in the first direction and a plurality oflines connecting the pixels in the second direction; and aninterpolating means for executing a first interpolation operation at theintersecting points used as the reference at the time of the selectionwith respect to each set of selected original pixel data, executing asecond interpolation operation in the first direction with respect tothe obtained plurality of interpolated data, and generating new pixeldata to be displayed on the display means based on a correspondencebetween the address and the position information obtained from themapping means.

According to a third aspect of the present invention, there is providedan image projection apparatus having a display means having displaypixels arranged in a matrix in first and second directions orthogonal toeach other and a projecting means for projecting an image displayed onthe display means to a projection surface by utilizing light from alight source and having a function of converting an input original imageto an image having distortion corrected in accordance with an angle ofthe projection with respect to a normal line of the projection surfacewhen projecting the image to the projection surface, comprising anaddress generating means for finding first interpolation addresses by afirst relationship equation with a coefficient of a coordinate parameterof the first direction in coordinates based on the pixel position of theoriginal image set as “1”, finding second interpolation addresses by asecond relationship equation with a coefficient of a coordinateparameter in the second direction set as “1” and generating addresses ofthe image suffering from distortion; a mapping means for linkingpositional information of the original image without distortion with theaddresses of the image suffering from distortion; and an interpolatingmeans for finding positions of intersecting points between an addressline of the image suffering from distortion generated by the addressgenerating means corresponding to the displayed pixels in the firstdirection and a plurality of lines connecting the original pixels in thesecond direction by using the first interpolation addresses, executingthe first interpolation operation at the intersecting points, executingthe second interpolation operation at the interpolation points found byusing the second interpolation addresses with respect to the obtainedplurality of interpolated data, and generating new pixel data to bedisplayed on the display means based on correspondences of the addressesobtained from the mapping means.

According to a fourth aspect of the present invention, there is providedan image conversion method comprising a first interpolation step ofrepeatedly executing a first interpolation operation by a plurality oforiginal pixel data arranged in one direction of either the vertical orhorizontal direction of an input original image; a data storage step oftemporarily storing a plurality of interpolated data generated by thefirst interpolation operation in a storing means; a second interpolationstep of generating new pixel data by executing a second interpolationoperation with respect to a plurality of the interpolated data in theother direction different from the one direction; and a step ofgenerating new pixel data by repeating the first interpolation step, thedata storage step, and the second interpolation step, wherein, in thestep of generating the new pixel data, if the combination of pluralityof the original pixel data is the same as a combination used whencalculating the interpolated data already stored in the storing means,the interpolated data is read out from the storing means and used forthe second interpolation operation.

According to a fifth aspect of the present invention, there is providedan image conversion method for converting an input original imageprojected onto a projection surface by utilizing light to an imagehaving distortion corresponding to an angle of the projection correctedon the projection surface by using interpolation processing andoutputting the same to a display means, comprising a step of generatingaddresses of the image suffering from distortion; a step of mappinglinking pixel positions of the original image without distortion withthe addresses of the image suffering from distortion; a step ofselecting a plurality of original image data for every intersectingpoint based on intersecting points between an address line of the imagesuffering from distortion generated by the address generating meanscorresponding to the display position of the display means in a firstdirection between the horizontal and vertical directions and a pluralityof lines connecting the pixels in a second direction different from thefirst direction; a step of executing a first interpolation operation atthe intersecting points used as the reference at the time of selectionfor each set of selected original image data; and a step of executing asecond interpolation operation in the horizontal direction with respectto a plurality of interpolated data obtained by the first interpolationoperation and generating new pixel data to be displayed on the displaymeans based on a correspondence of addresses obtained by the mapping.

According to a sixth aspect of the present invention, there is providedan image conversion method comprising an address generation step ofgenerating first interpolation addresses by a first relationshipequation with a coefficient of a coordinate parameter of a firstdirection between the horizontal and vertical directions set as “1” andgenerating a second interpolation address by a second relationshipequation with a coefficient of a coordinate parameter of a seconddirection different from the first direction set as “1”; a firstinterpolation step of selecting a plurality of original pixel dataarranged in the second direction of the input original image by usingthe first interpolation addresses and repeatedly executing the firstinterpolation operation a plurality of times; and a second interpolationstep of selecting a plurality of interpolated data arranged in the firstdirection generated by the first interpolation operation by using thesecond interpolation addresses, executing a second interpolationoperation at the interpolation points, and generating new pixel data.

BRIEF DESCRIPTION OF THE INVENTION

FIG. 1 is a view of arrangement of a front projector seen from above atthe time of front projection in first to third embodiments of thepresent invention.

FIG. 2 is a view of a range of possible arrangement of a projector inthe first to third embodiments of the present invention centered aroundthe position of front arrangement.

FIG. 3A is a view of the time of image projection from the side; FIG. 3Bis a view of an input image; and FIG. 3C is a view of an image on an LCDpanel surface.

FIG. 4A is a view of the time of oblique projection from a positionoffset in both the horizontal and vertical directions from the frontposition; FIG. 4B is a view of an input image; and FIG. 4C is view of animage on the panel surface of LCD.

FIG. 5 is a view of the basic configuration of a projector in the firstto third embodiments of the present invention.

FIG. 6 is a block diagram of an example of the configuration of an imageprocessor and peripheral circuits thereof included in the circuitportion of FIG. 5 in a projector of the first to third embodiments ofthe present invention.

FIG. 7 is a block diagram of an example of the configuration of acircuit inside the image processor.

FIG. 8 is a block diagram of the configuration of a filter unit in thefirst and second embodiments of the present invention.

FIG. 9A is a view of positional relationships between the projector anda screen in a right hand coordinate system in the case of frontprojection; FIG. 9B is a yz plan view; and FIG. 9C is an xy plan view.

FIG. 10A is a view of positional relationships between the projector anda screen in a right hand coordinate system in the case of a verticalprojection angle of α degrees and a horizontal projection angle of βdegrees; FIG. 10B is a yz plan view; and FIG. 10C is an xy plan view.

FIG. 11A to FIG. 11C are views of positional relationships of obliqueprojection equivalent to FIG. 10A to FIG. 10C where the screen isaxially rotated.

FIG. 12A and FIG. 12B are views of coordinate relationships together.

FIG. 13A is a view of an image of an address map of an SVGA output imageof the front projection; and FIG. 13B is a view of a distorted imageresulting from keystone deformation.

FIG. 14A is a view obtained by superimposing two images at the time ofthe mapping; and FIG. 14B is a display screen of a LCD panel generatedby an interpolation operation.

FIG. 15A-1 to FIG. 15B-2 are an xy plan view and a yz plan view of aright hand coordinate system at the time of projection from the bottomright toward the screen and plan views equivalent to them.

FIG. 16A-1 to FIG. 16B-2 are an xy plan view and a yz plan view of aright hand coordinate system at the time of projection from the top lefttoward the screen and plan views equivalent to them.

FIG. 17A-1 to FIG. 17B-2 are an xy plan view and a yz plan view of aright hand coordinate system at the time of projection from the topright toward the screen and plan views equivalent to them.

FIG. 18 is a view obtained by superimposing address lines of thekeystone distorted image on the original image.

FIG. 19 is an explanatory view of a case of generating two pixel data.

FIG. 20 is a block diagram of a filter unit showing the content of aregister after a data shift.

FIG. 21 is an explanatory view of the generation of image data by aninterpolation method of a second embodiment of the present invention.

FIG. 22 is a block diagram of a filter unit in a third embodiment.

FIG. 23A is a view of an image obtained by mapping an image withoutdistortion at addresses of a distorted image at the time of obliqueprojection; FIG. 23B is a view of an image of the address map limitingan interpolation region; and FIG. 23C is a view of an image of theaddress map after the interpolation operation in the vertical direction.

FIG. 24 is an explanatory view of a vertical interpolation position forevery line.

FIG. 25A is an explanatory view used for deriving vertical interpolationaddresses (address coordinate system of the original image) taking noteof one line; and FIG. 25B is a view of the address coordinate systemafter the mapping.

FIG. 26A is a view of an image of the address map of a verticalinterpolated image; FIG. 26B is a view of an image of the address mapafter image combination; and FIG. 26C is a view of an image of theaddress map after horizontal interpolation.

FIG. 27A and FIG. 27B are explanatory views of horizontal interpolationfor generating one pixel data.

FIG. 28A-1 to FIG. 28B-2 are views of an original image at the time oftrapezoidal distortion correction in the horizontal direction, the imageafter the conversion, and the image after the address conversion.

FIG. 29 is a view of an example of a one-dimensional interpolationoperation for generating one pixel data by a convolution operation by afour-tap filter.

FIG. 30 is a block diagram of the general configuration of a filter unitfor performing a one-dimensional filter operation independently twotimes.

BEST MODE FOR WORKING THE INVENTION

Embodiments of an image projection apparatus (projector) and an imageconversion method and apparatus of the present invention will beexplained next by referring to the drawings.

FIG. 1 shows a view of the arrangement of a front projector seen fromabove at the time of front projection.

In FIG. 1, the short broken line on the lower side shows the center lineof a projector body, while an upper long broken line is a lineconnecting the center of the lens in the projector and the center of thescreen.

A projector 1 and a screen 101 are arranged so that the center axis ofthe projection light indicated by the long broken line and theprojection surface on which the video is projected, for example, thescreen 101, are orthogonal when seen from above. The video projected bythe projector 1 is a television video signal or signal of a computerscreen. The shape of the display region of the video superimposed onthese signals is a square having a ratio of the length of the sides(aspect ratio) of 4:3 or 16:9 as the overall video, although there is adifference in the number of pixels according to the signal, as can beunderstood also from a television receiver and a computer display. Ifthe square video displayed on the LCD panel of the projector 1 is notstraightly projected, the projected video will also not become squareand consequently the original shape of the video will be distorted.

FIG. 2 is a view of a range where the projector 1 can be arranged in anembodiment of the present invention where the center position of theprojector arranged at the front is P0.

The projector 1 can be arranged in a horizontal plane Ph including thefront position P0 and can be arranged in a vertical plane Pv includingthe front position P0. Further, the projector 1 can be freely arrangedin any of a first quadrant ZONE1, a second quadrant ZONE2, a thirdquadrant ZONE3, and a fourth quadrant ZONE4 defined by the two planes Phand Pv.

The projector 1 can project the image of the LCD panel inside it fromany position so far as the position is within the above mentioned range.The projector 1 has a function of correcting the distortion of the imagein accordance with the projection position. Accordingly, when it is madeto function in this way, the image of a regular square having the sameaspect ratio as that when projected from exactly the front surface canbe projected onto the screen 101. This correction will be referred to as“keystone distortion correction”.

FIG. 3A shows an image projected from the left side toward the screen inthe horizontal plane. FIG. 3B shows the input image; while FIG. 3C showsthe image on the LCD panel surface built into the projector.

As shown in FIG. 3A, the projector 1 is arranged and projects an imageat the left side toward the screen 101, but the video on the screen 101appears as if the image were projected from the front. Originally, theprojected screen would have been deformed with the entirety distorted toa trapezoid including the hatching portion in the figure. This will bereferred to as a “horizontal keystone deformation”. Correction of thehorizontal keystone deformation will be referred to as “horizontalkeystone correction”.

In order to project an image from the projector 1 placed at side in thisway and project the video on the screen 101 as if it was projected fromthe front, it is necessary to calculate in advance how the image wouldbe distorted according to the projection position of the projector 1. Atthis time, by forming the image artificially distorted in the reversedirection with respect to the shape of distortion when the video isprojected from the side and projecting it, even if the video isprojected from the lateral direction, it can be made to appear the sameway as that at the time when the image is projected from the front. Inorder to obtain the projected video as in FIG. 3A in the above example,the input image of FIG. 3B is intentionally deformed on the LCD panelsurface as shown in FIG. 3C, and this image is projected onto the screen101.

FIG. 4A shows an image projected from the third quadrant ZONE3 in FIG.2. Further, FIG. 4B shows the input original image; while FIG. 4C showsthe image on the panel surface of the LCD.

The horizontal keystone distortion of FIG. 3A was trapezoidaldistortion, but in the case of FIG. 4A in which the distortion componentin the vertical direction is added to this, the distortion shape becomesfurther complex. When it is desired to obtain the regular squareprojection image after the correction shown in FIG. 4A, the LCD panelimage must be made an image as if the image were rotated in the LCDpanel surface as shown in FIG. 4C.

In both of the cases of FIG. 4C and FIG. 3C described above, if theimage intentionally distorted reversely to the projection image shapebefore the correction is displayed fully over the effective displayregion of the LCD panel surface, a correct square projection imagehaving the drop in the resolution and brightness suppressed as much aspossible is obtained on the screen.

Below, embodiments of the image projection apparatus and the imageprojection method able to perform such correction by converting theinput image to the image of the LCD panel will be explained in moredetail. In this image conversion, the general formula of the addressgeneration able to simultaneously correct horizontal and verticaldistortions is found taking as an example the case of projection fromthe third quadrant ZONE3 as shown in FIG. 4A. An image distorted in onlythe horizontal or only the vertical direction can be expressed by thecase where the horizontal or vertical projection angle is zero in thisgeneral formula. Further, the concept of projection from anotherquadrant other than the third quadrant ZONE3 is the same except only theformula is different.

First Embodiment

FIG. 5 shows the basic configuration of a projector.

The projector 1 has a circuit portion 2 including a circuit for applyingvarious signal processing to a video signal (input video signal) VIDEOand circuits of various driving systems. The circuit portion 2 includes,in part of the signal processing circuit, a central processing unit(CPU) 2 a as the control means and the processing means of the presentinvention, an address generation unit (ADG) 2 b as the addressgenerating means of the present invention, and a data interpolatingmeans (INT) 2 c. The projector 1 has a display means 3 b for displayingan image 3 a obtained by converting an original image indicated by asignal obtained by applying various signal processing to the input videosignal VIDEO, for example, an LCD panel. Further, the projector 1 has aprojecting unit 4 including a light source for illuminating the displaymeans 3 b and an optical unit 5 including various lenses for projectingthe image 3 a of the display means 3 b illuminated by the projectingunit 4. The LCD panel 3 may be any of a transmission type or areflection type. In either case, the image 3 a is projected on asubstantially perpendicular projection surface, for example, the screen101, through the optical unit 5 as a projected image 101 a. Three LCDpanels 3 are provided for the RGB colors. The images of the differentcolors are combined in the optical unit 5.

The CPU 2 a is one of the interpolating means in the broad sense sinceit controls the data interpolation. The CPU 2 a and the datainterpolating means 2 c constitute one embodiment of the “interpolatingmeans” in the present invention. The CPU 2 a acts also as a selectingmeans for selecting the original pixel data and a mapping means formapping for finding the relative relationship between addresses. The CPU2 a also has a role of controlling the rest of the configuration.Details of the computation and the mapping of the representative pointaddress will be explained later. Further, the “storing means” in thepresent invention is provided inside the data interpolating means 2 c,though not illustrated in the example illustrated in FIG. 5.

The projector 1 has a means 6 for getting the relative relationship dataindicating the relative relationship of images between the LCD panel 3and the screen 101 (hereinafter referred to as the “relativerelationship getting means”). The relative relationship getting (RRG)means 6 may take various forms, for example, an input unit for receivingas input the relative relationship data from the outside, an externaloperating means (button etc.), a storing means (for example, a ROM)storing the envisioned relative relationship data in advance, or a meansfor detecting the relative relationship by itself. The relativerelationship getting means 6 gets for example at least a distance of theimage up to the screen 101 and the angle formed by the optical axis ofthe optical unit 5 and the screen surface.

In a projector using a liquid crystal panel or other panel with fixedpixels, sometimes the number of pixels of the input original image andthe number of pixels of the output image will be different. For thisreason, the projector is provided with a signal processing function forconverting the number of pixels. This will be referred to as a “scalingfunction”. In this processing, data at a position where the pixel dataoriginally does not exist becomes necessary, so pixels are interpolated.In the interpolation operation, the pixel data of a target position isformed by using the data of the peripheral pixels. This function isrealized by building in for example a circuit block referred to as a“scaler” in an image processing circuit referred to as an “imageprocessor”.

FIG. 6 is a view of an example of the configuration of the imageprocessor included in the circuit portion 2 of FIG. 5 and the circuitblock on the periphery thereof.

The illustrated image processing circuit has a comb filter 21, a chromadecoder 22, a select switch (SW) 23, an analog-digital converter (A/DC)24, an image processor 25, an image memory 26 made of an SDRAM, etc.,and a CPU 2 a. Among these, the image processor 25 and the CPU 2 a areconcrete examples of the configuration for realizing the function ofimage conversion. Note that it is also possible to integrally form thefunctions of these image memory 26 and the CPU 2 a in the imageprocessor 25.

The illustrated image processing circuit can handle a video signal ofany of a composite video signal, a Y/C signal, and an RGB signal. Thecomposite video signal is input to the comb filter 21, the Y/C signal isinput to the chroma decoder 22, and the RGB signal is input to theselect switch 23. When considering the case where a composite videosignal is input, it is converted to the Y/C signal at the comb filter 21and converted to a YUV signal at the subsequent chroma decoder 22. Thesignal selected by the select switch 23 is converted by the A/DC 24 andbecomes a digital signal. This signal is input to the image processor25, where the desired signal processing is carried out. At this time,the processing of the image processor 25 is controlled by the CPU 2 a towhich the relative relationship information RRI is input. The imagememory 26 is appropriately used during the processing. After the desiredsignal processing is carried out, the processed signal is sent to thedisplay means, for example, the LCD panel 3. The image to be projectedis displayed on the LCD panel 3 based on this signal.

FIG. 7 shows an example of the configuration of the circuit block insidethe image processor.

The image processor 25 has an IP (interlace-progressive) conversion unit251, a scaler 252, a CPU interface 253, a memory control unit 254, and aread only memory (ROM) 255. The scaler 252 has an address generationunit 256 as an embodiment of the address generating means 2 b shown inFIG. 5, a coefficient generation unit 257, and a filter unit 258 as anembodiment of the data interpolating means 2 c shown in FIG. 5. Amongthem, the coefficient generation unit 257 and the filter unit 258 andthe CPU 2 a shown in FIG. 6 constitute an embodiment of the“interpolating means” in the present invention.

The video signal input to the image processor 25 is sent to the IPconversion unit 251, where an interlace signal is converted to aprogressive one. In this processing, the image memory 26 is used. By theconnection of the IP conversion unit 251 to the memory control unit 254as the memory interface, the IP conversion unit 251 transfers the imagedata with the image memory 26. The signal made progressive is sent tothe scaler 252 for the scaling. Inside the scaler 252, addressesnecessary for distortion correction are generated at the addressgeneration unit 256. The coefficient generation unit 257 is made togenerate filter coefficients and supply the generated filtercoefficients to the filter unit 258. The filter unit 258 performs theinterpolation operation using the given filter coefficients to convertthe original image indicated by the input video signal to an image ofthe LCD panel having the predetermined size and shape. The signal of theconverted image is output and sent to the LCD panel 3. The ROM 255 forholding the data such as the filter coefficients used in thisinterpolation operation is connected to the scaler 252. The interface253 of the CPU 2 a for controlling the image processing including thisseries of processing is connected to the IP conversion unit 251, thescaler 252, and the ROM 255.

FIG. 8 is a block diagram of the configuration of the filter unit 258.

The filter unit 258 includes a vertical direction interpolation filter(VIF) 11, a horizontal direction interpolation filter (HIF) 12, astoring means 13, and two selectors (SEL) 14 and 15. The storing means13 includes a first register 13 a and a second register 13 b.

The original data and the vertical direction set of filter coefficientsare input to the vertical direction interpolation filter 11. The outputof the vertical direction interpolation filter 11 is connected to theinput of the selector 14, while the output of the selector 14 isconnected to the first register 13 a. The output of the first register13 a is connected to the input of the second register 13 b and the inputof the selector 15. The output of the selector 15 is connected to thehorizontal direction interpolation filter 12. The horizontal directionset of filter coefficients is input to the horizontal directioninterpolation filter 12. New pixel data is output from the horizontaldirection interpolation filter 12.

When the original pixel data and the set of filter coefficient are firstinput to the vertical direction interpolation filter 11, the requiredoriginal pixel data is selected by the CPU 2 a and the selected originalpixel data is passed through the vertical direction interpolation filter11, whereby a one-dimensional (vertical direction) filter operation isexecuted. The interpolation filter 11 outputs interpolated datagenerated by using for example four pixel data in the verticaldirection. This is repeated at for example four pixel trains at theperiphery of an interpolation point, whereby four interpolated data Va,Vb, Vc, and Vd in the vertical direction are sequentially output fromthe vertical direction interpolation filter 11 to the selector 14. Theselector 14 distributes the four vertical direction interpolated dataVa, Vb, Vc, and Vd to predetermined storage portions (or predeterminedaddresses) in the first register 13 a while switching the output. Whenfour interpolated data Va, Vb, Vc, and Vd are gathered, the firstregister 13 a outputs them to the second register 13 b and the selector15. The four interpolated data Va, Vb, Vc, and Vd pass through theselector 15 and are input to the horizontal interpolation filter 12. Thehorizontal direction interpolation filter 12 executes a one-dimensional(horizontal direction) filter operation in accordance with the input setof filter coefficients and outputs the result as new pixel data.

In the same way as above, new pixel data are generated one afteranother. In order to generate one new bit of pixel data, four filteroperations in the vertical direction and one filter operation in thehorizontal direction are carried out. The second register 13 b has afunction of shifting the interpolated data. For this reason, the datatransfer is possible, for example, the interpolated data Vc and Vd areshifted up to the position where the interpolated data Va= and Vb= arestored in FIG. 8. Since it has such a shift function, in the second andsubsequent generations of the pixel data, the vertical direction filteroperation result (interpolated data Va= to Vd=) calculated at the timeof the one previous generation of pixel data can be set at the requiredposition of the second register 13 b.

The CPU 2 a controls the entire filter unit 258 including theseregisters 13 a and 13 b and selectors 14 and 15. Further, the CPU 2 acontrols the filter unit 258 and the coefficient generation unit 257 soas to control the vertical direction interpolation filter 11 so as notto continuously perform processing using the same filter coefficients bythe same combination of original images. Instead of this, the verticalinterpolation operation result (interpolated data) calculated theprevious time is shifted to the required position in the second register13 b and given via the selector 15 to the horizontal directioninterpolation filter 12 under the control of the CPU 2 a.

Next, an explanation will be given of the address generation of adistorted image.

The relative relationship data from the relative relationship getting(RRG) means 6 is input to the CPU 2 a (FIG. 6). The CPU 2 a itself andfurther the address generation unit 256 in the image processor 25 shownin FIG. 7 under control of the CPU 2 a generate the addresses of thedistorted image data for efficiently converting an original image.

Below, an explanation will be mainly given of the case where the imageof the front projector is obliquely projected from a position orientedupward by α degrees in the vertical direction and rotated to the left byβ degrees from the screen front surface in the horizontal directionbased on the position of the front surface with respect to the screen.

When the angles α and β are positive, the projection position falls inthe third quadrant ZONE3 of FIG. 2. Correction at the time of projectionfrom another quadrant can be carried out by substantially the sameconcept and method. At this time, an explanation will be given of thecase of also performing correction for eliminating distortion of aprojected image on a screen by image conversion processing when a videosignal having a resolution of VGA (640 pixels×480 lines) is input as theinput signal, this is converted in resolution to SVGA (800 pixels×600lines), and this is projected from an oblique direction.

FIG. 9A shows the positional relationship between the projector 1 andthe screen 101 in a right hand coordinate system of the case of frontprojection. Further, a yz plan view corresponding to that is shown inFIG. 9B, while an xy plan view is shown in FIG. 9C. At this time, thepositional coordinates of the projector are expressed by (Px, Py, Pz),and the positional coordinates of the any point on the screen 101 areexpressed by (Sx, Sy, Sz). The distance between the screen 101 and theprojector 1 determined by the positional coordinates (Px, Py, Pz) and(Sx, Sy, Sz), and the oblique projection angles α and β are the relativerelationship data mentioned above.

As shown in FIG. 9B and FIG. 9C, in front projection, the screen surfaceand the optical axis intersect perpendicularly. However, the opticalaxis intersects with the screen surface not at the center of the screen,but at a position closer to the bottom, i.e., here, near the center ofthe bottom side of the screen. This is because where a front projectoris arranged on a desk or arranged so as to be suspended from a ceiling,the two are arranged so that the line connecting the center of the lensand the center of the screen does not become parallel to the ground.This is the specification designed so as not to project the bottom endpart of the image to be projected onto the desk when the projector isplaced on the desk and projected and referred to as “optical offset”.

FIG. 10A shows the positional relationship between the projector and thescreen in a right hand coordinate system where the image is projectedobliquely upward with an angle by α degrees in the vertical directionand by β degrees with respect to the screen from the left in thehorizontal direction. Further, FIG. 10B shows a yz plan view, and FIG.10C shows an xy plan view. At this time, the rotation angle of the righthand coordinate system becomes α degrees in the vertical direction and(−β) degrees in the horizontal direction.

Here, the change of the relative viewpoint for facilitatingunderstanding of the keystone distortion correction will be considered.In FIG. 10A to FIG. 10C, the image was projected from an obliquedirection by moving the position of the projector 1, but here it isassumed that the screen 101 is axially rotated at that position withoutmoving the projector 1 while maintaining the relative positionalrelationship.

FIG. 11A to FIG. 11C show positional relationships of the obliqueprojection equivalent to FIG. 10A to FIG. 10C but where the screen isaxially rotated. At this time, when expressed in a right hand coordinatesystem, the screen is inclined by (−α) degrees in the y-direction (backside) from the standing position centered around the bottom side thereof(x-axis) and rotated by β degrees in the counterclockwise directioncentered around the left side (z-axis). That is, the inclinations andthe angles of rotation shown in FIG. 11B and FIG. 11C have signs inversewith respect to the projection angles from the positions of arrangementof the projector shown in FIG. 10A to FIG. 10C.

Below, as shown in FIG. 11A to FIG. 11C, it will be considered how theprojected video (projected image) is deformed when there is a projector1 at the front position and the video is projected to the screen (below,described as 101 t) inclined from this position.

The light projected by the projector 1 passes through the zx plane inwhich the screen 101 was located in FIG. 10A and is projected onto theinclined screen. The flat plane in which the inclined screen 101 t islocated is rotated by (−α) degrees in the vertical direction centeredaround the origin and rotated by β degrees horizontally, therefore canbe expressed using a rotation matrix centered around the origin. In thepresent embodiment, there is rotation in the horizontal direction andthe vertical direction, therefore the rotation matrix is defined by theprocedure of first rotation in the horizontal direction, then rotationin the vertical direction. Specifically, a normal vector (nx, ny, nz) onthe zx plane is expressed by the following matrix formula (2) byrotation:

$\begin{matrix}{\begin{pmatrix}{nx} \\{ny} \\{nz}\end{pmatrix} = {\begin{pmatrix}1 & 0 & 0 \\0 & {\cos\left( {- \alpha} \right)} & {- {\sin\left( {- \alpha} \right)}} \\0 & {\sin\left( {- \alpha} \right)} & {\cos\left( {- \alpha} \right)}\end{pmatrix}\begin{pmatrix}{\cos\;\beta} & {{- \sin}\;\beta} & 0 \\{\sin\;\beta} & {\cos\;\beta} & 0 \\0 & 0 & 1\end{pmatrix}\begin{pmatrix}0 \\1 \\0\end{pmatrix}}} & (2)\end{matrix}$

When considering a straight line connecting the position of theprojector 1 and the point on the zx plane where the screen 101 islocated and finding the intersecting point between this straight lineand a plane having the normal vector of the matrix formula (1), thecoordinate point projected on the plane of the inclined screen 101 t isfound. When viewing this inclined coordinate point focusing at the frontof the screen as shown in FIG. 10A, it is sufficient to rotate by αdegrees in the vertical direction and (−β) degrees in the horizontaldirection centered about the origin as the rotation in the oppositedirection again. Then, the shape of distortion where the image isprojected from the oblique direction is found. The coordinates in thex-direction, the y-direction, and the z-direction derived by such amethod are shown in equations (3-1), (3-2), and (3-3).

$\begin{matrix}{{Kx} =} & \left( {3\text{-}1} \right) \\{\mspace{76mu}\frac{{{\left( {{{Sx} \cdot {Pz}} - {{Sz} \cdot {Px}}} \right) \cdot \sin}\;\alpha} + {{\left( {{{Sy} \cdot {Px}} - {{Sx} \cdot {Py}}} \right) \cdot \cos}\;\alpha}}{\begin{matrix}{{{\left( {{Sx} - {Px}} \right) \cdot \sin}\;\left( {- \beta} \right)} + {{\left( {{Sy} - {Py}} \right) \cdot \cos}\;{\alpha \cdot \cos}\;\left( {- \beta} \right)} -} \\{{\left( {{Sz} - {Pz}} \right) \cdot \sin}\;{\alpha \cdot \cos}\;\left( {- \beta} \right)}\end{matrix}}\;} & \; \\{\mspace{79mu}{{Ky} = 0}} & \left( {3\text{-}2} \right) \\{{Kz} =} & \left( {3\text{-}3} \right) \\{\mspace{59mu}\frac{\begin{matrix}{{{\left( {{{Sx} \cdot {Py}} - {{Sy} \cdot {Px}}} \right) \cdot \sin}\;{\alpha \cdot \sin}\;\left( {- \beta} \right)} + {{\left( {{{Sx} \cdot {Pz}} - {{Sz} \cdot {Px}}} \right) \cdot \cos}\;{\alpha \cdot}}} \\{{\sin\;\left( {- \beta} \right)} + {{\left( {{{Sy} \cdot {Pz}} - {{Sz} \cdot {Py}}} \right) \cdot \cos}\;\left( {- \beta} \right)}}\end{matrix}}{\begin{matrix}{{{\left( {{Sx} - {Px}} \right) \cdot \sin}\;\left( {- \beta} \right)} + {{\left( {{Sy} - {Py}} \right) \cdot \cos}\;{\alpha \cdot \cos}\;\left( {- \beta} \right)} -} \\{{\left( {{Sz} - {Pz}} \right) \cdot \sin}\;{\alpha \cdot \cos}\;\left( {- \beta} \right)}\end{matrix}}} & \;\end{matrix}$

The (Kx, Ky, Kz) represented by these equations are coordinates deformedby keystone distortion when projecting an image by orienting theprojector 1 upward by α degrees vertically as shown in FIG. 10B and atan angle of (−β) degrees from the left with respect to the screen 101 inthe horizontal direction as shown in FIG. 10C.

FIG. 12A and FIG. 12B show the coordinate relationships all together. Inthese diagrams, (Sx, Sy, Sz) are the coordinates of the screen andcorrespond to the coordinates of the original image projected on thescreen to the regular square shape in the case of front projection.Further, the coordinates (Kx=, Ky=, Kz=) are the coordinates projectedon the plane of the screen 101 t obliquely inclined as in FIG. 11B andFIG. 11C. As explained above, (Kx, Ky, Kz) are coordinates deformed dueto the keystone distortion.

In this way,the above three equations (3-1), (3-2), and (3-3) givedeformed coordinates resulting from keystone distortion caused byprojection from any direction.

Next, the deformed coordinates resulting from equations (3-1), (3-2),and (3-3) of the coordinates matched with the resolution of the outputsignal (video) to the LCD panel 3 serving as the display means arefound. That is, in the case of SVGA output, the x-coordinate Sx of theimage before the distortion changes from 0 to 799, and the z-coordinateSz changes from 0 to 599. The x-coordinate Kx and the z-coordinate Kzafter the keystone distortion at this time are found. Note that they-coordinates Sy and Ky are zero since the image is located in the zxplane.

FIG. 13A shows an image PI_(OUT) of the address map of the SVGA outputimage of front projection, and FIG. 13B shows an image (below, distortedimage) PI_(K) of the address map of the SVGA output image after thekeystone deformation converted in coordinates while defining α=10 and−β=−30. In these diagrams, in order to avoid complexity of theillustration, sampling points of all of the pixel positions are notshown. The sampling points are represented by one dot for every 33pixels. Addresses of these sampling points can be calculated at the CPU2 a when necessary or addresses calculated in advance can be provided asa reference table in the ROM 255. In the latter case, the ROM 255corresponds to the relative relationship getting means 6 shown in FIG.1.

Next, as shown in FIG. 14A, the image PI of the video to be obtained bythe correction (virtual image of the projected image to be realized onthe screen, below referred to as a “projected image”) is superimposed onthe coordinate space deformed due to the distortion shown in FIG. 13B.Due to this, the projected image PI is mapped on the distorted imagePI_(K) and the correspondences of the addresses of the two images aredetermined. At this time, the input original image is VGA, but to adjustthe size and position of the image, the projected image PI can bearranged with any size (for example, size of SVGA) and at any positionin the deformed address space (distorted image PI_(K)). Note that if theprojected image PI is not completely contained in the distorted imagePI_(K), after interpolation, part of the image will be missing.Accordingly, desirably it is required that the size of a projected imagehaving the desired aspect ratio (4:3 in the present example) become themaximum limit in the address space of the distorted image. Then, theposition and the size of this projected image PI devolve to a simplegraphic issue. For example, with the position and the size as in FIG.14A, the relationship between the projected image PI and the distortedimage PI_(K) is uniquely determined.

In such mapping (linking addresses), the address distribution of thedistorted image PI_(K) is already found from the above equations (3-1),(3-2), and (3-3). Therefore, even if an actual physical memory (storageresources) is not used, for example the mapping can be executed assuminga virtual memory space in the CPU 2 a. For this reason, the mappingitself becomes high speed and there is no transfer of data with aphysical memory, therefore even if redone several times, the ratio ofthe total time of the processing in the entire time of the imageconversion is extremely small.

The correspondence of the addresses obtained by the mapping is addresscorrespondence between the distorted image and the desired projectedimage which becomes a regular square on the screen without distortion,but the distorted image is a result of projection of the image on theLCD panel of a regular square originally without distortion.Accordingly, by utilizing the above correspondence of addresses, theimage of the LCD panel 3 for obtaining the projected image on the screenwithout distortion can be generated.

In more detail, in the case of SVGA output, the number of thecoordinates of the effective display region of the LCD panel 3 becomes800×600. For all of these points, the interpolation is carried out atthe mapped addresses of the image. Among the interpolations at the800×600 points at this time, in the interpolations at all addresses ofthe region in which the distorted image. PI_(K) and the projected imagePI overlap shown in FIG. 14A, the filter coefficients are selected so asto be able to reproduce the image data like the projected image, and aplurality of pixel data of the original image required for thereproduction of the image are weighted by the filter coefficients toform new pixel data. The formed pixel data are allocated to theaddresses uniquely determining the arrangement position in the SVGAscreen as a regular square screen having an aspect ratio of 4:3 based onthe above address correspondence found by mapping. On the other hand, inthe interpolation of the region in the distorted image PI_(K) at theperiphery of the projected image PI shown in FIG. 14A, the resultbecomes combination of black colored pixels having no image data,therefore black colored image data is allocated to the correspondingpositions in the SVGA screen even after interpolation.

FIG. 14B shows an SVGA output image generated by such an imageconversion procedure. This image is an image after keystone distortioncorrection. When displaying this image on the LCD panel as in FIG. 4Cand projecting it as in FIG. 4A, a regular square projected image isobtained on the screen. The correspondences of addresses obtained by themapping are determined so that the area in which the image sizes overlapbecomes the maximum as mentioned above, therefore, in the projectedimage on the screen, the drops in brightness and resolution aresuppressed to a minimum.

In the above explanation of FIG. 13A and FIG. 13B, the output imagePI_(OUT) was given by 600×800 addresses matching with the size of theoutput image (SVGAn image), this was deformed, and the distorted imagePI_(K) was generated. Then, as shown in FIG. 14A, a procedure ofsuperimposing the projected image PI on the distorted image whilechanging the size and the position of the projected image PI and findingthe address correspondence required for the distortion correction fromthe two images after superimposition was employed. An addresscorrespondence the same as this can be found also by the followingtechnique.

In FIG. 14A, the output image PI_(OUT) is given by 64×480 addresses thesame as the original image (VGAn image) and is deformed to generate thedistorted image PI_(K). In FIG. 14A, the size of the distorted imagePI_(K) is changed, while the size of the projected image PI is notchanged as it corresponds to the SVGA from the first. Only the positionthereof is changed for the optimum superimposition of the two images. Bythis technique as well, the result becomes the same as that of FIG. 14A.

While distortion correction at the time of projection from the bottomleft position (third quadrant ZONE3) toward the screen was explainedabove, in the case of projection from another position, only theequations for finding the distortion coordinates are different. Theprocedure of the correction method explained above is the same.

FIG. 15A-1 and FIG. 15B-1 are an xy plan view and a yz plan view of aright hand coordinate system at the time of projection from the bottomright position (fourth quadrant ZONE4) toward the screen 101. Whenassuming that the projection position of the projector 1 is not movedfrom front projection, the xy plan view and the yz plan view in the casewhere the screen 101 is axially rotated so that the same relativerelationship is obtained are shown in FIG. 15A-2 and FIG. 15B-2. At thistime, the rotation angle of the right hand coordinate system becomes(−α) degrees in the vertical direction and (−β) degrees in thehorizontal direction.

The equations for finding the keystone distortion coordinates projectedon the inclined screen 101 t are shown in equations (4-1), (4-2), and(4-3):

$\begin{matrix}{{Kx} = \frac{{{\left( {{{Sx} \cdot {Pz}} - {{Sz} \cdot {Px}}} \right) \cdot \sin}\;\alpha} + {{\left( {{{Sy} \cdot {Px}} - {{Sx} \cdot {Py}}} \right) \cdot \cos}\;\alpha}}{{{\left( {{Sx} - {Px}} \right) \cdot \sin}\;\beta} + {{\left( {{Sy} - {Py}} \right) \cdot \cos}\;{\alpha \cdot \cos}\;\beta} - \mspace{95mu}{{\left( {{Sz} - {Pz}} \right) \cdot \sin}\;{\alpha \cdot \cos}\;\beta}}} & \left( {4\text{-}1} \right) \\{\mspace{79mu}{{Ky} = 0}} & \left( {4\text{-}2} \right) \\{{Kz} = \frac{{\left( {{{Sz} \cdot {Py}} - {{Sy} \cdot {Px}}} \right)\sin\;{\alpha \cdot \sin}\;\beta} + {{\left( {{{Sz} \cdot {Pz}} - {{Sz} \cdot {Pz}}} \right) \cdot \mspace{45mu}\cos}\;{\alpha \cdot \sin}\;\beta} + {{\left( {{{Sy} \cdot {Pz}} - {{Sz} \cdot {Py}}} \right) \cdot \cos}\;\beta}}{{{\left( {{Sx} - {Px}} \right) \cdot \sin}\;\beta} + {{\left( {{Sy} - {Py}} \right) \cdot \cos}\;{\alpha \cdot \mspace{25mu}\cos}\;\beta} - {{\left( {{Sz} - {Pz}} \right) \cdot \sin}\;{\alpha \cdot \cos}\;\beta}}} & \left( {4\text{-}3} \right)\end{matrix}$

FIG. 16A-1 and FIG. 16B-1 are an xy plan view and a yz plan view of aright hand coordinate system at the time of projection from the top leftposition (second quadrant ZONE2) toward the screen 101. Further, whenthe projection position of the projector 1 is not moved from the frontprojection, the xy plan view and the yz plan view in the case where thescreen 101 is axially rotated so that the same relative relationship isobtained are shown in FIG. 16A-2 and FIG. 16B-2. At this time, therotation angle of the right hand coordinate system becomes α degrees inthe vertical direction and becomes β degrees in the horizontaldirection.

The equations for finding the keystone distortion coordinates projectedon the inclined screen 101 t are shown in equations (5-1), (5-2), and(5-3).

$\begin{matrix}{{Kx} = \frac{{\left( {{{Sx} \cdot {Pz}} - {{Sz} \cdot {Px}}} \right) \cdot {\sin\left( {- \alpha} \right)}} + {\left( {{{Sy} \cdot {Px}} - {{Sx} \cdot {Py}}} \right) \cdot {\cos\left( {- \alpha} \right)}}}{{\left( {{Sx} - {Px}} \right) \cdot {\sin\left( {- \beta} \right)}} + {\left( {{Sy} - {Py}} \right) \cdot {\cos\left( {- \alpha} \right)} \cdot \mspace{11mu}{\cos\left( {- \beta} \right)}} - {\left( {{Sz} - {Pz}} \right) \cdot {\sin\left( {- \alpha} \right)} \cdot {\cos\left( {- \beta} \right)}}}} & \left( {5\text{-}1} \right) \\{\mspace{79mu}{{Ky} = 0}} & \left( {5\text{-}2} \right) \\{{Kz} = \frac{\begin{matrix}{{\left( {{{Sx} \cdot {Py}} - {{Sy} \cdot {Px}}} \right) \cdot {\sin\left( {- \alpha} \right)} \cdot {\sin\left( {- \beta} \right)}} + {\cdot \left( {{{Sx} \cdot {Pz}} - {{Sz} \cdot}} \right.}} \\{{\left. {Px} \right){{\cos\left( {- \alpha} \right)} \cdot {\sin\left( {- \beta} \right)}}} + {{\left( {{{Sy} \cdot {Pz}} - {{Sz} \cdot {Py}}} \right) \cdot \cos}\left( {- \beta} \right.}}\end{matrix}}{\begin{matrix}{{\left( {{Sx} - {Px}} \right) \cdot {\sin\left( {- \beta} \right)}} + {{\left( {{Sy} - {Py}} \right) \cdot {\cos\left( {- \alpha} \right)} \cdot \cos}\left( {- \beta} \right)} -} \\{\left( {{Sz} - {Pz}} \right) \cdot {\sin\left( {- \alpha} \right)} \cdot {\cos\left( {- \beta} \right)}}\end{matrix}}} & \left( {5\text{-}3} \right)\end{matrix}$

FIG. 17A-1 and FIG. 17B-1 are an xy plan view and yz plan view of aright hand coordinate system at the time of projection from the topright position (first quadrant ZONE1) toward the screen 101. Further,when the projection position of the projector 1 is not moved from frontprojection, the xy plan view and the yz plan view in the case where thescreen 101 is axially rotated so that the same relative relationship isobtained are shown in FIG. 17A-2 and FIG. 17B-2. At this time, therotation angle of the right hand coordinate system becomes α degrees inthe vertical direction and (−β) degrees in the horizontal direction.

The equations for finding the keystone distortion coordinates projectedon the inclined screen 101 t are shown in equations (5-1), (5-2), and(5-3).

$\begin{matrix}{{Kx} = \frac{{\left( {{{Sx} \cdot {Pz}} - {{Sz} \cdot {Px}}} \right) \cdot {\sin\left( {- \alpha} \right)}} + {\left( {{{Sy} \cdot {Px}} - {{Sx} \cdot {Py}}} \right) \cdot {\cos\left( {- \alpha} \right)}}}{{{{\left( {{Sx} - {Px}} \right) \cdot \sin}\;\beta} + {{\left( {{Sy} - {Py}} \right) \cdot {\cos\left( {- \alpha} \right)} \cdot \mspace{20mu}\cos}\;\beta} - {{\left( {{Sz} - {Pz}} \right) \cdot {\sin\left( {- \alpha} \right)} \cdot \cos}\;\beta}}\;}} & \left( {6\text{-}1} \right) \\{\mspace{76mu}{{Ky} = 0}} & \left( {6\text{-}2} \right) \\{{Kz} = \frac{\begin{matrix}{{{\left( {{{Sx} \cdot {Py}} - {{Sy} \cdot {Px}}} \right) \cdot {\sin\left( {- \alpha} \right)} \cdot \sin}\;\beta} + {\cdot \left( {{{Sx} \cdot {Pz}} - {{Sz} \cdot}} \right.}} \\{{\left. {Px} \right){{\cos\left( {- \alpha} \right)} \cdot \sin}\;\beta} + {{\left( {{{Sy} \cdot {Pz}} - {{Sz} \cdot {Py}}} \right) \cdot \cos}\;\beta}}\end{matrix}}{{{\left( {{Sx} - {Px}} \right) \cdot \sin}\;\beta} + {{\left( {{Sy} - {Py}} \right) \cdot {\cos\left( {- \alpha} \right)} \cdot \cos}\;\beta} - {{\left( {{Sz} - {Pz}} \right) \cdot {\sin\left( {- \alpha} \right)} \cdot \cos}\;\beta}}} & \left( {6\text{-}3} \right)\end{matrix}$

Next, a detailed explanation will be given of the interpolationoperation in the present embodiment.

FIG. 18 is a diagram superimposing the line connecting oblique addresspoints corresponding to a horizontal line among the addresses composingdeformed coordinates resulting from keystone distortion on the originalimage.

For convenience, as shown in FIG. 18, letters A, B, and C are attachedin the horizontal direction, numbers 1, 2, and 3 are attached in thevertical direction, and the original pixel data are indicated by thecombination of the horizontal and vertical positions at this time. Forexample, in FIG. 18, the original pixel data at the top left end isrepresented as “A1”. Now assume that the position of the pixel to beinterpolated is the position of the original pixel data of “B3”.

Below, an explanation will be given of the case of performing aninterpolation operation (first interpolation operation) on pixel data inthe vertical direction several times and performing an interpolationoperation (second interpolation operation) in the horizontal directionfor several interpolated data resulting from this.

In the present embodiment, whenever generating one new pixel data D1 byan interpolation operation, the combination of the original pixel dataused is selected. The selection is carried out based the intersectingpoints between the oblique address line AL1 and the vertical lines VLa,VLb, VLc, and VLd connecting the original pixels in the verticaldirection. Each of the vertical lines VLa, VLb, VLc, and VLd is a lineconnecting a constant position of the original pixel determined inadvance, for example, the pixel center. In FIG. 18, a pixel center isrepresented by a white circle. The intersecting point of the addressline AL1 and the vertical line VLa is defined by Pva. In the same way,the intersecting point of the address line AL1 and the vertical line VLbis defined by Pvb, the intersecting point of the address line AL1 andthe vertical line VLc is defined by Pvc, and the intersecting point ofthe address line AL1 and the vertical line VLd is defined by Pvd.

The intersecting point PVa lies between the center of the original pixeldata A3 and the center of the original pixel data A4, therefore theoriginal pixel data A2, A3, A4, and A5 are selected based on theintersecting point PVa. In the same way as above, the original pixeldata B2, B3, B4, and B5 are selected in the B column, the original pixeldata C1, C2, C3, and C4 are selected in the C column, and the originalpixel data D1, D2, D3, and D4 are selected in the D column.

The original pixel data is selected by the CPU 2 a shown in FIG. 6. Theselected base pixel data are sequentially input to the verticaldirection interpolation filter 11 of FIG. 8.

In the vertical direction interpolation filter 11, the firstinterpolation operation is sequentially executed. The firstinterpolation operation in the present example is a four-tap convolutionoperation. Accordingly, the vertical direction interpolated data Va, Vb,Vc, and Vd at the intersecting points PVa, PVb, PVc, and PVd arecalculated by using the computation equations shown in equations (7-1),(7-2), (7-3), and (7-4).

$\begin{matrix}{{Va} = {{A\; 2 \times {h\left( {{- 1} - \frac{{za}\; 1}{{{za}\; 1} + {{za}\; 2}}} \right)}} + {A\; 3 \times {h\left( {- \frac{{za}\; 1}{{{za}\; 1} + {{za}\; 2}}} \right)}} + {A\; 4 \times {h\left( \frac{{za}\; 2}{{{za}\; 1} + {{za}\; 2}} \right)}} + {A\; 5 \times {h\left( {1 + \frac{{za}\; 2}{{{za}\; 1} + {{za}\; 2}}} \right)}}}} & \left( {7\text{-}1} \right) \\{{Vb} = {{B\; 2 \times {h\left( {{- 1} - \frac{{zb}\; 1}{{{zb}\; 1} + {{zb}\; 2}}} \right)}} + {B\; 3 \times {h\left( {- \frac{{zb}\; 1}{{{zb}\; 1} + {{zb}\; 2}}} \right)}} + {B\; 4 \times {h\left( \frac{{zb}\; 2}{{{zb}\; 1} + {{zb}\; 2}} \right)}} + {B\; 5 \times {h\left( {1 + \frac{{zb}\; 2}{{{zb}\; 1} + {{zb}\; 2}}} \right)}}}} & \left( {7\text{-}2} \right) \\{{Vc} = {{C\; 1 \times {h\left( {{- 1} - \frac{{zc}\; 1}{{{zc}\; 1} + {z\; c\; 2}}} \right)}} + {C\; 2 \times {h\left( {- \frac{{zc}\; 1}{{{zc}\; 1} + {{zc}\; 2}}} \right)}} + {C\; 3 \times {h\left( \frac{{zc}\; 2}{{{zc}\; 1} + {{zc}\; 2}} \right)}} + {C\; 4 \times {h\left( {1 + \frac{{zc}\; 2}{{{zc}\; 1} + {{zc}\; 2}}} \right)}}}} & \left( {7\text{-}3} \right) \\{{Vd} = {{D\; 1 \times {h\left( {{- 1} - \frac{{zd}\; 1}{{{zd}\; 1} + {{zd}\; 2}}} \right)}} + {D\; 2 \times {h\left( {- \frac{{zd}\; 1}{{{zd}\; 1} + {{zd}\; 2}}} \right)}} + {D\; 3 \times {h\left( \frac{{zd}\; 2}{{{zd}\; 1} + {{zd}\; 2}} \right)}} + {D\; 5 \times {h\left( {1 + \frac{{zd}\; 2}{{{zd}\; 1} + {{zd}\; 2}}} \right)}}}} & \left( {7\text{-}4} \right)\end{matrix}$

Here, ineEquation (7-1), the phase difference za1 is the phasedifference between the original pixel center of the data A3 and theintersecting point PVa, while the phase difference za2 is the phasedifference between the original pixel center of the data A4 and theintersecting point PVa. In the same way, in equation (7-2), the phasedifference zb1 is the phase difference between the original pixel centerof the data B3 and the intersecting point PVb, and the phase differencezb2 is the phase difference between the original pixel center of thedata B4 and the intersecting point PVb. These phase differences areshown in FIG. 18. Also, the phase differences zc1, zc2, zd1, and zd2between the other intersecting points PVc and PVd and the centers of twopixels nearest in the vertical direction are defined in the same way asabove. Based on these phase differences, the vertical direction set offilter coefficients is generated by the coefficient generation unit 257shown in FIG. 7. Note that it is also possible to generate the verticaldirection set of filter coefficients in advance and hold it in the ROM255 etc.

Due to this, the vertical direction interpolated data Va, Vb, Vc, and Vdare generated at the positions of the intersecting points PVa, PVb, PVc,and PVd. The vertical direction interpolated data Va, Vb, Vc, and Vd arestored at predetermined addresses of the first register 13 a while beingswitched by the selector 14 shown in FIG. 8.

Next, the interpolated data Va, Vb, Vc, and Vd pass through the selector15 and are supplied to the horizontal direction interpolation filter 12.The horizontal direction interpolation filter 12 executes a horizontaldirection interpolation operation (second interpolation operation) onthe interpolated data Va, Vb, Vc, and Vd.

In the second interpolation operation, the phase difference between thetwo vertical lines VLb and VLc nearest in the horizontal direction andthe interpolation position P1 are defined by x1 and x2. At this time,the pixel data D1 generated by the horizontal interpolation at theinterpolation position P1 is calculated by the four-tap interpolationoperation shown in equation (8).

$\begin{matrix}{{D\; 1} = {{{Va} \times {h\left( {{- 1} - \frac{x\; 1}{{x\; 1} + {x\; 2}}} \right)}} + {{Vb} \times {h\left( {- \frac{x\; 1}{{x\; 1} + {x\; 2}}} \right)}} + {{Vc} \times {h\left( \frac{x\; 2}{{x\; 1} + {x\; 2}} \right)}} + {{Vb} \times {h\left( {1 + \frac{x\; 2}{{x\; 1} + {x\; 2}}} \right)}}}} & (8)\end{matrix}$

Note that, the horizontal direction set of filter coefficients isgenerated at the coefficient generation unit 257 based on the phasedifferences x1 and x2 and supplied from the coefficient generation unit257 to the horizontal direction interpolation filter 12 directly orafter passing through the ROM 255 etc.

As described above, new pixel data D1 was generated at the interpolationposition P1 by the plurality of (four in the present example) firstinterpolation operations and one second interpolation operation. In thesame way as the above, pixel data D2 is generated at the nextinterpolation position P2.

FIG. 19 shows the relationship between the address line and the originalimage when generating pixel data at the position P2 after the positionP1.

When generating data at the position P2 by interpolation processing,from the periphery thereof, four bits of data in four columns, i.e., 16bits of original image data in total, are selected. In the same way asthe case of generating the position P1, the original image data isselected using the intersecting points between the oblique address lineAL1 and the vertical direction lines of the four columns as thereference. For example, when selecting the set of the original pixeldata so that two pixel centers are located at one side and the otherside of each intersecting point in the vertical direction, in theexample shown in FIG. 19, the original pixel data of row numbers 1 to 4are selected up to the C column, the D column, and the E column, but theoriginal pixel data of row numbers 0 to 3 are selected in the F column.At this time, the original pixel data of the C column and the D column,that is, the set of the original pixel data (C1, C2, C3, C4) and the setof the original pixel data (D1, D2, D3, D4), are the same as those whencalculating the pixel data the previous time. Accordingly, reuse of thefirst interpolation operation result (interpolated data Vc, Vd) usingthe set of the same original pixel data is possible.

In FIG. 8, the interpolated data Va=, Vb=, Vc=, and Vd= stored in thesecond register 13 b are the interpolated data used for the calculationof the pixel data D1 the previous time. These data were stored in thefirst register 13 a first, but are transferred to the second register ata predetermined timing. During the period from the start of thehorizontal interpolation operation for generating the pixel data D1 theprevious time to the end of the vertical interpolation operation forgenerating the pixel data D2 this time, the held content of the secondregister 13 b is shifted to the left side in FIG. 8 by exactly therequired amount. In the example of FIG. 19, the interpolated data Vc′and Vd′ corresponding to the C column and the D column are shifted tothe position of the interpolated data Va′ and Vb′ in FIG. 8.

FIG. 20 is a block diagram of a filter unit showing the register contentafter this shift.

Originally, four interpolation operations in the vertical direction areexecuted for calculating new pixel data, but in the present embodiment,operations on interpolated data which can be reused are omitted. In moredetail, under the control of the CPU 2 a, the interpolation operationsof the C column and the D column are omitted, and the interpolationoperations of the E column and the F column are executed by the verticaldirection interpolation filter 11. The interpolated data Ve and Vfobtained as a result are stored as illustrated in the first register 13a under the control of the selector 14.

Next, under the control of the selector 15, the interpolated data Vc′and Vd′ corresponding to the C column and the D column are read from thesecond register 13 b, and the interpolated data Ve and Vf correspondingto the E column and the F column are read from the first register 13 a.When the horizontal direction interpolation filter 12 performs thesecond interpolation operation by using these interpolated data, the newpixel data D2 is generated at the position P2.

In the present embodiment, in the generation of the pixels of the secondand subsequent times in this way, the first interpolation operationresult used for the generation of the pixel data immediately before thiscan be reused. This becomes possible since the original pixel data usedfor the first interpolation operation is selected based on theintersecting points between the oblique address line and the verticallines.

That is, in the usual method of uniformly deciding the range of theoriginal pixel data as to for example 4×4 centered around theinterpolation position, one new pixel data is indeed generated, but thevertical interpolation operation ends up being performed four times.Accordingly, even when there is almost no horizontal distortioncomponent, the same computation is uselessly repeated in many cases.Further, even if trying to reuse the calculation result, if the range ofthe original pixel data is uniformly determined, no sets of overlappingoriginal pixel data will be generated if there is even a littleinclination of the address line and therefore reuse becomessubstantially impossible.

In the present embodiment, the selection range of the original pixeldata flexibly changes in accordance with the inclination of the addressline, therefore a higher efficiency of the first interpolation operationhaving a computation time several times that of the second interpolationoperation can be achieved. The reduction of the number of times of thecomputation of the first interpolation operation is directly connectedto the improvement of the computation efficiency of the whole, thereforethe efficiency of generation of the image for correcting distortion canbe effectively improved.

Due to the above, the processing time can be reduced in the signalprocessing necessary when a projector projects an image from an obliquedirection. Further, since the overlapped vertical direction interpolateddata is not recomputed, the pixel data fetched from the memory necessaryoverlappingly in the vertical direction operation unit becomesunnecessary. For this reason, without greatly changing the interpolationtechnique, the load of the processing of the memory is reduced and thebit size of the memory may be made small.

Note that the number of pixels in the vertical direction and the numberof pixels in the horizontal direction used for the interpolationoperation are not limited to four and can be arbitrarily set. Further,it is also possible to change the method of the selection of theoriginal image data in accordance with the inclination of the addressline. For example, when the inclination of the address line AL1 shown inFIG. 18 is smaller, the pixel data of the same row number is selectedfrom the A column to the C column, and the pixel data is selected withina range obtained by an upward shift by exactly the amount of one pixelfor only the D column, while when the inclination of the address lineAL1 is larger than that of the case of FIG. 18, it is possible to selectthe pixel data within a range obtained by shifting three columns of theB column to the D column upward by exactly the amount of one pixel withrespect to the pixel data selected in the A column.

Second Embodiment

In the second embodiment, conversely to the first embodiment, the firstinterpolation operation is carried out in the horizontal direction, andthe second interpolation operation is carried out in the verticaldirection. As the filter unit, in FIG. 8, use can be made of oneobtained by switching the positions of the vertical interpolation filter11 and the horizontal interpolation filter 12.

FIG. 21 shows the relationship between an address line and the originalimage when generating the pixel data D1 at the position P1 by theinterpolation method of the second embodiment.

The address line AL1 of the image suffering from distortion overlaps theoriginal image as illustrated. New pixel data is found at the point P1on the address line. In this case, based on the intersecting pointsbetween the address line AL1 and the lines connecting constant positionsdetermined in advance of the original image, for example, the centers ofthe pixels, that is, the horizontal lines HL2, HL3, HL4, and HL5, a setof a plurality of, i.e., four, original image data arranged in thehorizontal direction is selected. When based on the intersecting pointPH2, the set of the original image data (C2, D2, E2, F2) is selected.When based on the intersecting point PH3, the set of the original imagedata (B3, C3, D3, E3) is selected, when based on the intersecting pointPH4, the set of the original image data (A4, B4, C4, D4) is selected,and when based on the intersecting point PH5, the set of the originalimage data (A5, B5, C5, D5) is selected.

The interpolated data H2, H3, H4, and H5 in the horizontal direction atthe intersecting points PH2, PH3, PH4, and PH5 are calculated by usingthe computation equations shown in equations (9-1), (9-2), (9-3), and(9-4):

$\begin{matrix}{{H\; 2} = {{C\; 2 \times {h\left( {{- 1} - \frac{x\; 21}{{x\; 21} + {x\; 22}}} \right)}} + {D\; 2 \times {h\left( {- \frac{x\; 21}{{x\; 21} + {x\; 22}}} \right)}} + {E\; 2 \times {h\left( \frac{x\; 22}{{x\; 21} + {x\; 22}} \right)}} + {F\; 2 \times {h\left( {1 + \frac{x\; 22}{{x\; 21} + {x\; 22}}} \right)}}}} & \left( {9\text{-}1} \right) \\{{H\; 3} = {{B\; 3 \times {h\left( {{- 1} - \frac{x\; 31}{{x\; 31} + {x\; 32}}} \right)}} + {C\; 3 \times {h\left( {- \frac{x\; 31}{{x\; 31} + {x\; 32}}} \right)}} + {D\; 3 \times {h\left( \frac{x\; 32}{{x\; 31} + {x\; 32}} \right)}} + {E\; 3 \times {h\left( {1 + \frac{x\; 32}{{x\; 31} + {x\; 32}}} \right)}}}} & \left( {9\text{-}2} \right) \\{{H\; 4} = {{A\; 4 \times {h\left( {{- 1} - \frac{x\; 41}{{x\; 41} + {x\; 42}}} \right)}} + {B\; 4 \times {h\left( {- \frac{x\; 41}{{x\; 41} + {x\; 42}}} \right)}} + {C\; 4 \times {h\left( \frac{x\; 42}{{x\; 41} + {x\; 42}} \right)}} + {D\; 4 \times {h\left( {1 + \frac{x\; 42}{{x\; 41} + {x\; 42}}} \right)}}}} & \left( {9\text{-}3} \right) \\{{H\; 5} = {{A\; 5 \times {h\left( {{- 1} - \frac{x\; 51}{{x\; 51} + {x\; 52}}} \right)}} + {B\; 5 \times {h\left( {- \frac{x\; 51}{{x\; 51} + {x\; 52}}} \right)}} + {C\; 5 \times {h\left( \frac{x\; 52}{{x\; 51} + {x\; 52}} \right)}} + {D\; 5 \times {h\left( {1 + \frac{x\; 52}{{x\; 51} + {x\; 52}}} \right)}}}} & \left( {9\text{-}4} \right)\end{matrix}$

Here, in equation (9-1), the phase difference x21 is the phasedifference between the original pixel center of the data D2 and theintersecting point PH2, while the phase difference x22 is the phasedifference between the original pixel center of the data E2 and theintersecting point PH2. In the same way, in equation (9-4), the phasedifference x51 is the phase difference between the original pixel centerof the data B5 and the intersecting point PH5, while the phasedifference x52 is the phase difference between the original pixel centerof the data C5 and the intersecting point PH5. These phase differencesare shown in FIG. 21. Also the phase differences x31, x32, x41, and x42between the other intersecting points PH3 and PH4 and two original pixelcenters nearest in the horizontal direction are defined in the same wayas above.

Due to this, the interpolated data H2, H3, H4, and H5 in the horizontaldirection are generated at the positions of the intersecting points PH2,PH3, PH4, and PH5.

The interpolated data H2, H3, H4, and H5 in the horizontal direction areheld once in the first register 13 a.

Thereafter, the vertical direction interpolation filter executes avertical direction interpolation operation (second interpolationoperation) on the interpolated data H2, H3, H4, and H5.

In the second interpolation operation, the phase differences between twohorizontal lines HL3 and HL4 nearest in the vertical direction and theinterpolation position P1 are defined as z1 and z2. At this time, thepixel data D1 generated by the vertical interpolation at theinterpolation position P1 is calculated by the four-tap interpolationoperation equation shown in equation (10).

$\begin{matrix}{{D\; 1} = {{H\; 2 \times {h\left( {{- 1} - \frac{z\; 1}{{z\; 1} + {z\; 2}}} \right)}} + {H\; 3 \times {h\left( {- \frac{z\; 1}{{z\; 1} + {z\; 2}}} \right)}} + {H\; 4 \times {h\left( \frac{z\; 2}{{z\; 1} + {z\; 2}} \right)}} + {H\; 5 \times {h\left( {1 + \frac{z\; 2}{{z\; 1} + {z\; 2}}} \right)}}}} & (10)\end{matrix}$

As described above, even if performing the horizontal directioninterpolation operation first and performing the vertical directioninterpolation operation on the results, the new pixel data D1 can begenerated in the same way as above. Note that the generation of thesubsequent pixel data and the control of the register at that time(reuse of the interpolated data) are carried out in the same way as thefirst embodiment.

Third Embodiment

In the above first and second embodiments, each time calculating one newpixel data, a plurality of first interpolation operation and secondinterpolation operation were repeated.

In the third embodiment, first all of the first interpolation operationscorresponding to for example an address line are ended and then thesecond interpolation operations are repeatedly executed.

FIG. 22 is a block diagram of a filter unit having the configurationsuitable for executing the interpolation method in the third embodiment.

The filter unit 258 illustrated in FIG. 22 has a vertical interpolationfilter 11, a horizontal interpolation filter 12, and a line memory 16.The line memory 16 has a storage capacity able to store at least oneline having the maximum resolution at one time. The line memory 16 iscontrolled by controlling for example a not illustrated drive circuit bythe CPU 2 a.

The deformation coordinates (Kx, Ky, Kz) due to the keystone distortionfound by equations (3-1) to (3-3) shown in the first embodiment are forsimultaneously finding the addresses of the interpolation points of thex-coordinates and the z-coordinates. Accordingly, this mode is notrequired for finding new pixel data with a high efficiency by theinterpolation method of repeating the first interpolation operation inone direction of the vertical direction or the horizontal direction asin the interpolation method of the present embodiment and performing thesecond interpolation operation in the other direction of the verticaldirection or the horizontal direction with respect to a plurality ofinterpolated data obtained as the result of this.

For this reason, below, the addresses of the distorted image shown inequation (3-1) and equation (3-3) are converted to the formats ofaddresses which can be independently interpolated in the verticaldirection or the horizontal direction. That is, each of the keystonedeformation addresses in the x-direction indicated by equation (3-1) andeach of the keystone deformation addresses in the z-direction indicatedby (3-3) is converted to two addresses, that is, an address of theinterpolation position used in the vertical interpolation and an addressof the interpolation position used in the horizontal interpolation. As aresult, any address (x, z) of the keystone deformation coordinates isdivided to the two sets of vertical direction elements (VX, VZ) and ahorizontal direction elements (HX, HZ). However, the horizontaldirection elements are at equal intervals at the time of verticalinterpolation, while the vertical direction elements become equal ininterval at the time of horizontal interpolation. Therefore, the equalinterval elements can be used as they are. Accordingly, in practice, oneset of addresses of a combination of (VX, HZ) or (HX, VZ) is calculatedat a time. It is possible to calculate the addresses at the CPU 2 a orcalculate them by the CPU 2 a in advance and store them in the ROM 255or the like as a table.

First, the vertical direction interpolation will be explained as thefirst interpolation operation. In the present embodiment, the verticaldirection interpolation operation is continuously carried out by exactlythe required number of times corresponding to the number of pixels inthe horizontal direction.

FIG. 23A shows the mapping image when projecting an image to the screenwith an angle of 30 degrees from the left side and 10 degrees from thebottom when facing the screen. This mapping image is found in the sameway as FIG. 14A found in the first embodiment. The following explanationwill exemplify the case where the size and the position of the imageafter the interpolation are adjusted by changing the size and theposition of the image to be interpolated with respect to the addressesof the keystone distortion given by equations (3-1) and equation (3-3).

FIG. 24 is a diagram showing an enlarged one part of FIG. 23B in whichthe address line AL1 found from Equation (3-1) and Equation (3-3) andthe original image overlap.

The address line AL1 is the path of the positions at which a series ofone line=s worth of original pixels in the horizontal direction isconverted in address by keystone distortion in the same way as the casesof the first and second embodiments. In the case of the projection fromthe bottom left side when facing the screen, it forms a straight lineinclined in the oblique direction. Each intersecting point between theoblique address line AL1 and the vertical lines connecting the originalpixels in the vertical direction is indicated by a mark “x” in FIG. 24.These intersecting points, that is, the intersecting point PVa of the Acolumn, the intersecting point PVb of the B column, the intersectingpoint PVc of the C column, the intersecting point PVd of the D column,the intersecting point PVe of the E column, the intersecting point PVfof the F column, the intersecting point PVg of the G column, theintersecting point PVh of the H column, . . . become the interpolationaddresses at the time of the vertical direction interpolation operationas the first interpolation operation.

In the third embodiment, these interpolation addresses are calculatedwithout using complex calculation equations such as equations (3-1) to(3-3).

First, in the present embodiment, in order to simply find theinterpolation addresses, as shown in FIG. 23B, the range of thecomputation region in the horizontal direction is limited to the rangewhere the addresses of the distorted image and the regular square image(original image) overlap. Further, in order to simplify the computationequation of the addresses, coordinate conversion becomes necessary.

FIG. 25A shows an address map of an original image before the keystonedeformation and the xz coordinate axes at that time. FIG. 25B shows theaddress map after the keystone deformation, but the original xzcoordinate axes have as the origin the left corner P0 of the keystonedeformed image. That is, equation (3-1) and equation (3-3) arecomputation equations at coordinates using this P0 as the origin.

In the present embodiment, as shown in FIG. 25B, the xz coordinates areobtained using as the origin the left corner of the regular squareoriginal image overlapping the address map after the keystonedeformation. In these xz coordinates, the z-coordinate of the address ofany point is represented by the x-coordinate. This is because, thehorizontal direction coordinate (x-coordinate) of an interpolationaddresses used in vertical interpolation becomes a discrete value havingunits of a constant value for every pixel, so the computation equationis simplified by doing this.

In more detail, when expressing the coordinates of an address invertical interpolation by (VX, VZ), in one line=s worth of interpolationoperations, the x-coordinates VX of the interpolation addresses changein discrete values in units of constant values for every pixel in thehorizontal direction. Further, the z-coordinates VZ at that time are onthe oblique address line AL1 to which the interpolation pixels belong onthe address map of the keystone deformation, therefore can be calculatedby finding the inclination. The inclination of the address line AL1becomes (KzeBKzs)/(KxeBKxs) using the address coordinates (Kxs, Kzs) atwhich the start point pixel of the line of the original image is locatedafter the keystone deformation and the address coordinates (Kxe, Kze) atwhich the end point pixel is located after the keystone deformation asshown in FIG. 25A and FIG. 25B. When this inclination is used, thez-coordinates VZ of the interpolation addresses can be expressed as inequation (11).

$\begin{matrix}{{VZ} = {{Kzs} + {\left( {{VX} - {Kxs}} \right) \times \frac{\left( {{Kze} - {Kzs}} \right)}{\left( {{Kxe} - {Kxs}} \right)}}}} & (11)\end{matrix}$

Accordingly, a vertical interpolation address (VX, VZ) of a pixelarranged at the position (x, z) after the interpolation in the xzcoordinate system of FIG. 25B becomes as in equation (12). VX changes atequal intervals for every number of horizontal pixels of the input image(original image):

$\begin{matrix}{{\left( {{VX},{VZ}} \right) = {\text{(}x}},{{Kzs}_{z} + {\left( {x - {Kxs}_{z}} \right) \times \frac{\left( {{Kze}_{z} - {Kzs}_{z}} \right)}{\left( {{Kxe}_{z} - {Kxs}_{z}} \right)}}}} & (12)\end{matrix}$

In equation (12), as shown in FIG. 25B, a keystone deformation addressobtained by equations (3-1) and (3-3) with respect to the start pointpixel at which the oblique address line intersects the z-axis isexpressed by (Kxs_(z), Kzs_(z)), while a keystone deformation addressobtained by equations (3-1) and (3-3) of the end point pixel at whichthe oblique address line intersects a line parallel to the z-axiscorresponding to the right side of the region of FIG. 23B is expressedby (Kxe_(z), Kze_(z)).

Based on the vertical direction interpolation address indicated byequation (12), a plurality of original pixel data arranged in thevertical direction are selected for every column. In this selection, inthe same way as the first embodiment, for example the CPU 2 a selectstwo upper and lower pixel data each, i.e., four original image data intotal, for every interpolation address. The selected basic pixel dataare sequentially input to the vertical interpolation filter 11 of FIG.22.

Due to this, the interpolated image as in FIG. 23C can be obtained. Asan example, assumed vertical interpolation on the interpolation positionPVf of FIG. 24. At this time, when the phase ratio between theinterpolation position PVf and predetermined positions of the upper andlower pixels, for example, the pixel centers, is z1:z2, the value of theinterpolated data Vf is computed as in equation (13).

$\begin{matrix}{{Vf} = {{F\; 2 \times {h\left( {{- 1} - \frac{z\; 1}{{z\; 1} + {z\; 2}}} \right)}} + {F\; 3 \times {h\left( {- \frac{z\; 1}{{z\; 1} + {z\; 2}}} \right)}} + {F\; 4 \times {h\left( \frac{z\; 2}{{z\; 1} + {z\; 2}} \right)}} + {F\; 5 \times {h\left( {1 + \frac{z\; 2}{{z\; 1} + {z\; 2}}} \right)}}}} & (13)\end{matrix}$

As shown in equation (13), one interpolated data Vf is generated byusing two original pixel data F2 and F3 above the interpolation positionand two original pixel data F4 and F5 below it. At the otherinterpolation positions PVa to PVe, PVh, . . . , the interpolated datais found by the same interpolation operation. As a result, the samenumber of the interpolated data as the horizontal resolution of theoriginal image are output from the vertical interpolation filter 11 ofFIG. 22 one after another and sequentially input to the line memory 16.The data held in the line memory 16 is used in the next horizontalinterpolation processing (second interpolation operation). Note thatdepending on the processing, the storage capacity of the line memory 16is not limited to the amount one line and may be a plurality of linesworth of capacity.

FIG. 26A shows the vertically interpolated image. Next, verticalinterpolation processing is applied for the image-less regions at theright and left cut off when limiting the region in FIG. 23A to FIG. 23B.The compressed parts are added to the interpolation image of FIG. 26A.The images are combined in the line memory 16 resulting in the image ofFIG. 26B.

Horizontal direction interpolation processing (second interpolationoperation) is carried out on the image of FIG. 26B. Using theinterpolation address at this time as (lix, Hz), one image isinterpolated. In this case, the vertical direction lines have alreadybeen interpolated, so to change the interpolation address Hz in thevertical direction for every vertical line, the coefficient of thez-coordinate thereof is made “1”. Further, for the horizontal addresslix, use is made of the initial address calculated in equation (3-1).Accordingly, the interpolation address (LIX, HZ) used in the horizontaldirection interpolation of a pixel arranged at the position of (x, z)after the interpolation becomes as in equation (14).

$\begin{matrix}\left. {{\left( {{HX},{HZ}} \right) = \left( \frac{{{\left( {{x \cdot {Pz}} - {z \cdot {Px}}} \right) \cdot \sin}\;\alpha} + {\left( {{- X} \cdot {Py}} \right)\cos\;\alpha}}{{\left( {x - {Px}} \right) \cdot {\sin\left( {- \beta} \right)}} + {{\left( {- {Py}} \right) \cdot \cos}\;{\alpha \cdot {\cos\left( {- \beta} \right)}}} - {{\left( {z - {Pz}} \right) \cdot \sin}\;{\alpha \cdot {\cos\left( {- \beta} \right)}}}} \right)},z} \right) & (14)\end{matrix}$

Here, the value of the horizontal element HX of an interpolation addressis the x-coordinate Kx of the keystone deformation shown in equation(3-1) and designates the interpolation address in the x-direction usedin the mapping as shown in FIG. 23A. The y-coordinate Sy of theprojection surface in equation (3-1) of the x-coordinate Kx of thekeystone deformation was denoted as zero since the screen is on the zxplane.

In this way, the horizontal direction interpolation address (HX, HZ) isfound. The horizontal direction interpolation address (HX, HZ) is inputto the horizontal direction interpolation filter 12 from the addressgeneration unit or the ROM etc. The horizontal direction interpolationfilter executes the horizontal direction interpolation operation at eachpoint of the interpolation address in the corresponding address line byusing the vertical direction interpolated data held in the line memory16.

FIG. 27A and FIG. 27B are explanatory views of the horizontalinterpolation for generating a certain pixel data.

As shown in FIG. 27B, when generating the pixel data at for example theposition P1, use is made of four interpolated data Vc, Vd, Ve, and Vfselected from the interpolated data having the smallest phase differenceon the address line AL1 based on this horizontal direction interpolationposition P1. When the phase difference between the horizontal directioninterpolation position P1 and the position PVd of the interpolated dataVd at this time is x1 and the phase difference between the horizontaldirection interpolation position P1 and the position PVe of theinterpolated data Ve is x2, the second interpolation operation equationbased on a convolution operation of these four pixel data become asshown in equation (15).

$\begin{matrix}{{D\; 1} = {{{Vc} \times {h\left( {{- 1} - \frac{x\; 1}{{x\; 1} + {x\; 2}}} \right)}} + {{Vd} \times {h\left( \frac{x\; 1}{{x\; 1} + {x\; 2}} \right)}} + {{Ve} \times {h\left( \frac{x\; 2}{{x\; 1} + {x\; 2}} \right)}} + {{Vf} \times {h\left( {1 + \frac{x\; 2}{{x\; 1} + {x\; 2}}} \right)}}}} & (15)\end{matrix}$

The new pixel data D1 generated by this is generated, whereby theinterpolation processing of one pixel is completed. By performing thisinterpolation processing in the horizontal direction with respect to allhorizontal direction pixels while appropriately changing theinterpolated data selected, one line's worth of the image subjected tokeystone correction can be generated. Further, by performing the sameprocessing on the other lines and performing all of the processingincluding the interpolation processing in the vertical direction and thehorizontal direction in one frame, one frame of the corrected image isgenerated.

In the explanation of the third embodiment, the order of theinterpolation operations was described as performing verticalinterpolation as the first interpolation processing, then performinghorizontal interpolation as the second interpolation processing, but itis also possible to perform horizontal interpolation as the firstinterpolation processing for exactly at least one line=s worth of theframe first, then perform the vertical interpolation as the secondinterpolation processing. Further, it is also possible to sequentiallyperform the first interpolation processing and the second interpolationprocessing in units of several lines or sequentially perform the firstinterpolation processing and the second interpolation processing inunits of a frame.

In the above explanation, in the work of mapping the image with theaddresses of the keystone distortion for the keystone correction, theposition of the finally prepared keystone corrected image was adjustedby adjusting the size and the position of the image without changing theaddresses. In the present embodiment, it is also possible to makeadjustments by relatively changing the values of the addresses whilefixing the size and the position of the image. In this case, theinterpolation addresses found by equation (12) and equation (14) may beexpressed by equations changing according to the change of the size andthe position of the image.

In the third embodiment, after performing the first interpolationoperation for at least one line=s worth of a frame, the secondinterpolation operation is repeated for at least the same number oftimes as the horizontal resolution. At this time, the coefficients ofeither the vertical direction or the horizontal direction coordinateparameters of the interpolation addresses used for the firstinterpolation operation are made “1”, while the other of the verticaldirection or the horizontal direction coordinate parameters of theinterpolation addresses used for the second interpolation operation aremade “1”. By this, in the vertical direction interpolation operation,processing maintaining the intervals of the pixels in the horizontaldirection as they are for the positions of the pixels after the mappingand interpolating in only the vertical direction in accordance with themapping of the keystone distortion becomes possible. Conversely, in thehorizontal direction interpolation operation, processing maintaining theintervals of the pixels in the vertical direction as they are for thepositions of the pixels after the mapping and interpolating in only thehorizontal direction in accordance with the mapping of the keystonedistortion becomes possible.

As an effect common to the first to third embodiments, the range ofselection of the original image data flexibly changes in accordance withthe inclination of the address line, so the number of times ofoverlapping useless operations is reduced. For this reason, theprocessing efficiency of the frequently performed first interpolationoperation can be improved.

Particularly, in the third embodiment, in addition to the reduction ofthe number of times of useless operations, as mentioned above, one setof addresses between each two sets of addresses originally generated forthe vertical interpolation and for the horizontal interpolation iseasily found by making the coefficients of the coordinate parameters“1”. For this reason, substantially the generation of one set of theaddresses is sufficient, and the efficiency of the address generation isimproved. Further, the interpolation address element not having thecoefficient of the coordinate parameter of “1” in an interpolationaddress used in the first interpolation operation performed previouslyis generated by a simple computation equation using the inclination ofthe address line. As a result, the efficiency of the first interpolationoperation performed several times more frequently than the secondinterpolation operation can be further advanced.

The reduction of the number of computations of the first interpolationoperation is directly linked with the improvement of the computationefficiency as a whole, so it becomes possible to effectively improve theefficiency of generation of an image for correcting the distortion.

From the above, the processing time can be reduced by the signalprocessing which becomes necessary when the projector projects an imagefrom the oblique direction. Further, not by recomputing the overlappingvertical interpolated data, the pixel data fetched from the memory,which had been necessary overlappingly in the vertical operation unit,becomes unnecessary. For this reason, the load of the processing of thememory is reduced without greatly changing the interpolation techniqueand also the bit size of the memory may be made smaller.

Note that, in the above embodiments, the explanation was given taking asan example use of the LCD panel 3 as the display means, but the presentinvention is not limited to this and can be widely applied so far as thedevice is a fixed pixel device in which display pixels are arranged in amatrix such as a DMD (digital micromirror device).

Further, in the above embodiments, the explanation was given of anexample wherein the projector 1 had the function of the image conversionprocessing for correcting the projection distortion, but the presentinvention is not limited to this and can be configured so as to correctthe projection distortion by outputting an image converted signal froman apparatus performing this image conversion processing to theprojector 1 too.

Further, in the above embodiments, the explanation was given of anexample in which the projection surface was substantially verticallyarranged, but the present invention is not limited to this. If theprojection distortion is corrected based on the angle of the projectionfrom the projector 1 with respect to the normal line of the projectionsurface, that is, the angle formed by the normal line of the projectionsurface and the optical axis of the optical unit 5, the projectionsurface may be arranged while being inclined with respect to thevertical line too. In this case, the coordinate conversion may becarried out for the arrangement of the screen 101 and the projector 1 bythe inclination angle thereof.

As explained above, the image conversion apparatus of the first aspectof the present invention comprises an interpolating means for receivingas input the original image data, executing a first interpolationoperation a plurality of times with respect to a plurality of originalpixel data arranged in one of a vertical or horizontal direction in theoriginal image, executing the second interpolation operation in theother direction, and generating new image data at the interpolationpoints and a storing means for storing the interpolated data obtained bythe first interpolation operation, wherein, where a combination of theplurality of original pixel data is the same as a combination previouslyused when calculating interpolated data already stored in the storingmeans, the interpolating means reads out that interpolated data from thestoring means and uses the same for the second interpolation operation.

Preferably, the image conversion apparatus is an image conversionapparatus which, when projecting an image onto a projection surface byutilizing light, converts an input original image to an image havingdistortion corrected in accordance with an angle of projection on theprojection surface by interpolation processing, which image conversionapparatus comprises an address generating means for generating addressesof the image suffering from the distortion and a mapping means forlinking positional information of the distortion-free original imagewith the addresses of the image suffering from distortion, wherein theinterpolating means generates the new pixel data to be output to thedisplay means based on the correspondences between the addresses and thepositional information obtained from the mapping means by the executionof a plurality of first interpolation operations and a secondinterpolation operation.

The image conversion method of the first aspect of the present inventionis for achieving the first object and comprises a first interpolationstep of repeatedly executing a first interpolation operation for aplurality of original pixel data arranged in either of a vertical orhorizontal direction of an input original image; a data storage step oftemporarily storing a plurality of interpolated data generated by thefirst interpolation operations in a storing means; a secondinterpolation step of generating new pixel data by executing a secondinterpolation operation with respect to the plurality of interpolateddata in the other direction different from the one direction; and a stepof generating new pixel data by repeating the first interpolation step,the data storage step, and the second interpolation step, wherein, inthe step of generating new pixel data, when a combination of a pluralityof original pixel data is the same as a combination previously used whencalculating the interpolated data and already stored in the storingmeans, that interpolated data is read out from the storing means andused for the second interpolation operation.

In the above image conversion apparatus, the address generating meansgenerates the addresses of the distorted image, and the addresses of thedistortion-free desired image are mapped on them. By this, thecorrespondences between the addresses of the interpolation points of thescreen of the display means and the pixel positions of the originalimage corresponding to the addresses are determined. Accordingly, thepixel data of the original image used for generating the new pixel dataat the interpolation points is also understood.

In the interpolation of the first aspect, the first interpolationoperation is carried out on a plurality of original pixel data arrangedvertically or horizontally, and the results thereof (interpolated data)are temporarily stored in the storing means. The second interpolationoperation is executed in a different direction with respect to theplurality of interpolated data. As a result, a new pixel data isgenerated by the interpolating means and output. The interpolated datain the storing means is held delayed until at least the time when it canbe utilized for the generation of other pixel data. After this,interpolated data having the same combination of the original image dataas interpolated data already generated and held is not newly generated,but is read out from the storing means and utilized for the secondinterpolation operation.

The image projection apparatus of the second aspect of the presentinvention is an image projection apparatus having a display means havingdisplay pixels and a projecting means for projecting an image of thedisplay means to a projection surface utilizing light from a lightsource and having a function of, when projecting an image, converting aninput original image to an image having distortion corrected inaccordance with the angle of projection on this projection surface byinterpolation processing, comprising an address generating means forgenerating addresses of an image suffering from distortion; a mappingmeans for linking positional information of a distortion-free originalimage with the addresses of the image suffering from distortion; aselecting means for selecting a plurality of original image data forevery intersecting point based on the intersecting points between anaddress line of the image suffering from distortion generatedcorresponding to the display pixels in the horizontal (vertical)direction and the plurality of lines connecting the original pixels inthe vertical (horizontal) direction; and an interpolating means forexecuting a first interpolation operation at the intersecting pointsused as the reference at the time of the selection for each set ofselected original pixel data, executing a second interpolation operationin a horizontal (vertical) direction with respect to the obtainedplurality of interpolated data, and generating new pixel data to beoutput to the display means based on the correspondences between theaddresses and the positional information obtained from the mappingmeans.

The image conversion method of the present invention is an imageconversion method which, when projecting an image to a projectionsurface by utilizing light, converts an input original image to an imagehaving distortion corrected in accordance with the angle of theprojection on a projection surface by interpolation processing andoutputs the same to a display means, comprising a step of generatingaddresses of an image suffering from distortion; a step of the mappingfor linking positional information of a distortion-free original imagewith addresses of the image suffering from distortion; a step ofselecting a plurality of original image data for every intersectingpoint based on the intersecting points between an address line on theimage suffering from distortion corresponding to the display position ofthe display means in the horizontal (vertical) direction and a pluralityof lines connecting the original pixels in the vertical (horizontal)direction; a step of executing a first interpolation operation at theintersecting points used as the reference at the time of selection foreach set of the selected original image data; and a step of executing asecond interpolation operation in the horizontal (vertical) directionwith respect to a plurality of interpolated data obtained by the firstinterpolation operation and generating new pixel data to be output tothe display means based on the correspondences of the addresses obtainedby the mapping.

In the above invention, the addresses of the image suffering fromdistortion are generated, mapping is performed between these addressesand positional information of an original image, then a plurality oforiginal pixel data arranged side in the vertical direction are selectedto be used for a first interpolation operation. At this time, based onthe intersecting points between an address line of the image sufferingfrom distortion and a plurality of lines connecting the original pixelsin the vertical direction, a plurality of original image data areselected for every intersecting point. Accordingly, when the addressline is oblique, a set of a plurality of original pixel data shifted inthe vertical direction in accordance with that can be selected. Evenwhen the inclination of the address line is particularly large, originalimage data suitable for maintaining the precision of the interpolationoperation is selected.

Next, a second interpolation operation is carried out on the pluralityof interpolated data obtained by the first interpolation operation,whereby new pixel data is generated.

The image projection apparatus of the third aspect of the presentinvention is an image projection apparatus having a display means havingdisplay pixels and a projecting means for projecting an image of thedisplay means to a projection surface by utilizing light from a lightsource and having a function of converting an input original image datato an image having distortion corrected in accordance with the angle ofthe on this projection surface when projecting by interpolationprocessing, comprising an address generating means for generating firstinterpolation addresses by a first relationship equation with acoefficient of a horizontal coordinate parameter in coordinates based onthe pixel position of the original image set as “1” and generatingsecond interpolation addresses by the second relationship equation witha coefficient of a vertical coordinate parameter set as “1”; a mappingmeans for linking positional information of a distortion-free originalimage with the addresses of the image suffering from distortion; and aninterpolating means for finding the positions of the intersecting pointsbetween an address line of the image suffering from distortion generatedcorresponding to the display pixels in the horizontal (vertical)direction and a plurality of lines connecting the original pixels in thevertical (horizontal) direction by using the first interpolationaddresses, executing the first interpolation operation at theseintersecting points, executing the second interpolation operation at theinterpolation points found by using the second interpolation addresses,and generating new pixel data to be displayed on the display means basedon the correspondences of addresses obtained from the mapping means.

The image conversion method of the present invention has an addressgeneration step of generating first interpolation addresses by a firstrelationship equation with a coefficient of a coordinate parameter of afirst direction among horizontal and vertical directions set as “1” andgenerating second interpolation addresses by a second relationshipequation with a coefficient of the coordinate parameter of the seconddirection different from the first direction set as “1”; a firstinterpolation step of selecting a plurality of original pixel dataarranged in the second direction of the input original image by usingthe first interpolation addresses and repeatedly executing a firstinterpolation operation a plurality of times; and a second interpolationstep of selecting a plurality of interpolated data arranged in the firstdirection generated by the first interpolation operation by using thesecond interpolation addresses and executing a second interpolationoperation at the interpolation points to generate new pixel data.

In the third aspect, an addresses found by the computation at the timeof the generation of addresses is only a combination of the other of thevertical and horizontal address of a first interpolation address inwhich the coefficient of the coordinate parameter is not “1” and onevertical or horizontal address of a second interpolation address. Anaddress at which the coefficient of the coordinate parameter is 1 is notcomputed, or even when it is computed, almost no load is placed on theaddress generating means. By using this simple address, when the firstinterpolation operation is repeatedly executed and the secondinterpolation operation is executed for a plurality of interpolated dataobtained as a result of this, a new pixel is generated in a short time.

According to the image conversion apparatus and image conversion methodaccording to the present invention, the load of the address computationand the load of the interpolation operation increasing due to anincrease of the address computation points or the complication of thecomputation equations are reduced and high speed processing becomespossible.

According to the image projection apparatus according to the presentinvention, the distortion of an image on a projection surface can becorrected at a high speed while effectively reducing the load on themeans for performing the address computation and the interpolationoperation and the load on the memory.

1. An image conversion apparatus comprising: an interpolating means forexecuting a first interpolation operation by a plurality of originalpixel data arranged in one of a vertical or horizontal direction in aninput original image, executing a second interpolation operation in theother direction different from said one direction using the plurality ofinterpolated data obtained by said first interpolation operation, andgenerating new image data at the interpolation points, and a storingmeans for storing said interpolated data obtained by said firstinterpolation operation, wherein, if a combination of said plurality oforiginal pixel data is the same as a combination previously used whencalculating said interpolated data already stored in said storing means,said interpolating means reads out that interpolated data from saidstoring means and uses the same for said second interpolation operation.2. An image conversion apparatus as set forth in claim 1, wherein saidinterpolating means successively generates new image data of saidinterpolation points arranged on a line inclined in a horizontaldirection or vertical direction with respect to said original pixels. 3.An image conversion apparatus as set forth in claim 1, furthercomprising a filter coefficient generating means for generating firstfilter coefficients to be used in said first interpolation operationwith respect to said plurality of original pixel data arranged in onedirection based on phases of points for finding said interpolated datawith respect to positions of said pixels arranged in one direction. 4.An image conversion apparatus as set forth in claim 3, wherein saidfilter coefficient generating means generates second filter coefficientsto be used in said second interpolation operation with respect to saidplurality of interpolated data arranged in the other direction based onphases of said interpolation points with respect to positions forfinding said interpolated data arranged in the other direction.
 5. Animage conversion apparatus as set forth in claim 2, wherein: saidstoring means comprises a memory for storing at least one line's worthof said plurality of interpolated data to be used for interpolation ofsaid interpolation points arranged in said inclined line, and saidinterpolating means repeatedly executes said first interpolationoperation for each line and repeatedly executes said secondinterpolation operation while changing the combination of interpolateddata selected from said one line's worth of interpolated data obtained.6. An image conversion apparatus as set forth in claim 5, wherein saidinterpolating means executes said first interpolation operations forindividual lines together for one image and executes said secondinterpolation operation using said one image's worth of interpolateddata obtained together for one image.
 7. An image conversion apparatusas set forth in claim 1, wherein said image conversion apparatuscomprises an image conversion apparatus which, when projecting an imageonto a projection surface by utilizing light, converts said originalimage to an image having distortion corrected in accordance with anangle of said projection with respect to a normal line of saidprojection surface by interpolation processing and outputs the result toa display means, which image conversion apparatus comprises: an addressgenerating means for generating addresses of the image suffering fromsaid distortion corresponding to display positions of said display meansand a mapping means for linking positional information of thedistortion-free original image with said addresses of the imagesuffering from distortion, wherein said interpolating means generatesthe new pixel data to be output to said display means based on thecorrespondences between said addresses and said positional informationobtained from said mapping means by the execution of a plurality of saidfirst interpolation operations using said storing means and said secondinterpolation operation.
 8. An image conversion apparatus as set forthin claim 7, wherein said addresses are generated by said addressgenerating means corresponding to matrix-like display pixels arranged inorthogonally intersecting first and second directions of said displaymeans, and said positional information of the original image is setcorresponding to said first and second direction pixels of said originalimage.
 9. An image conversion apparatus as set forth in claim 8, furthercomprising a selecting means for selecting a plurality of original imagedata for each intersecting point and outputting them to saidinterpolating means based on intersecting points between an address lineof an image suffering from distortion generated by said addressgenerating means corresponding to said display pixels arranged in saidone direction and a plurality of lines connecting said pixels in saidsecond direction.
 10. An image conversion apparatus as set forth inclaim 8, wherein said interpolating means executes said firstinterpolation operation at each intersecting point between an addressline of an image suffering from distortion generated by said addressgenerating means corresponding to said display pixels arranged in saidone direction and a plurality of lines connecting said pixels in saidsecond direction and executing said second interpolation operation onsaid obtained plurality of interpolated data.
 11. An image conversionapparatus as set forth in claim 10, wherein said interpolating meansrepeatedly executes said first interpolation operation for each addressline of said image suffering from distortion and repeatedly executessaid second interpolation operation while changing the combination ofinterpolated data selected from said one line's worth of interpolateddata obtained.
 12. An image conversion apparatus as set forth in claim11, wherein said interpolating means executes said first interpolationoperations for individual lines together for one image and executes saidsecond interpolation operation using said one image's worth ofinterpolated data obtained together for one image.
 13. An imageconversion apparatus as set forth in claim 11, wherein said addressgenerating means finds first interpolation addresses by a firstrelationship equation with a coefficient of a coordinate parameter ofsaid first direction in coordinates based on the pixel position of theoriginal image set as “1” and finds second interpolation addresses by asecond relationship equation with a coefficient of a coordinateparameter of said second direction set as “1” so as to generateaddresses of an image suffering from distortion.
 14. An image conversionapparatus as set forth in claim 13, wherein said first interpolationaddresses are found by said first relationship equation having as avariable only the coordinate parameter of said first direction changingat equal intervals.
 15. An image projection apparatus having a displaymeans having display pixels arranged in a matrix in first and seconddirections intersecting each other orthogonally and a projecting meansfor projecting an image displayed at the display means to a projectionsurface utilizing light from a light source and having a function of,when projecting an image to said projection surface, converting an inputoriginal image to an image having distortion corrected in accordancewith the angle of said projection to a normal line of said projectionsurface by interpolation processing, comprising an address generatingmeans for generating addresses of an image suffering from distortioncorresponding to positions displayed on said display means; a mappingmeans for linking positional information of a distortion-free originalimage with said addresses of the image suffering from distortion; aselecting means for selecting a plurality of original image data of saidsecond direction for every intersecting point based on the intersectingpoints between an address line of said image suffering from distortiongenerated by said address generating means corresponding to said displaypixels arranged in said first direction and the plurality of linesconnecting said pixels in said second direction; and an interpolatingmeans for executing a first interpolation operation at said intersectingpoints used as the reference at the time of the selection for each setof selected original pixel data, executing a second interpolationoperation in said first direction with respect to the obtained pluralityof interpolated data, and generating new pixel data to be output to thedisplay means based on the correspondences between the addresses andsaid positional information obtained from said mapping means.
 16. Animage projection apparatus having a display means having display pixelsarranged in a matrix in first and second directions intersecting eachother orthogonally and a projecting means for projecting an imagedisplayed at the display means to a projection surface utilizing lightfrom a light source and having a function of, when projecting an imageto said projection surface, converting an input original image to animage having distortion corrected in accordance with the angle of saidprojection to a normal line of said projection surface by interpolationprocessing, comprising an address generating means for finding firstinterpolation addresses by a first relationship equation with acoefficient of a coordinate parameter of said first direction incoordinates based on the pixel position of the original image set as “1”and finding second interpolation addresses by a second relationshipequation with a coefficient of a coordinate parameter of said seconddirection set as “1”; a mapping means for linking positional informationof a distortion-free original image with said addresses of the imagesuffering from distortion; and an interpolating means for finding thepositions of the intersecting points between an address line of saidimage suffering from distortion generated by said address generatingmeans corresponding to the display pixels of said first direction and aplurality of lines connecting the original pixels in said seconddirection by using said first interpolation addresses, executing thefirst interpolation operation at these intersecting points, executingthe second interpolation operation at the interpolation points found byusing the second interpolation addresses for the obtained interpolateddata, and generating new pixel data to be displayed on the display meansbased on the correspondences of addresses obtained from the mappingmeans.
 17. An image conversion method comprising: a first interpolationstep of repeatedly executing a first interpolation operation for aplurality of original pixel data arranged in either of a vertical orhorizontal direction of an input original image; a data storage step oftemporarily storing a plurality of interpolated data generated by thefirst interpolation operations in a storing means; a secondinterpolation step of generating new pixel data by executing a secondinterpolation operation with respect to the plurality of interpolateddata in the other direction different from said one direction; and astep of generating new pixel data by repeating the first interpolationstep, the data storage step, and the second interpolation step, wherein,in the step of generating new pixel data, if a combination of aplurality of original pixel data is the same as a combination previouslyused when calculating the interpolated data and already stored in thestoring means, that interpolated data is read out from the storing meansand used for the second interpolation operation.
 18. An image conversionmethod which, when projecting an image to a projection surface byutilizing light, converts an input original image to an image havingdistortion corrected in accordance with the angle of said projection ona projection surface by interpolation processing and outputs the same toa display means, comprising: a step of generating addresses of an imagesuffering from distortion; a step of mapping for linking positionalinformation of a distortion-free original image with addresses of theimage suffering from distortion; a step of selecting a plurality oforiginal image data for every intersecting point based on theintersecting points between an address line of an image suffering fromdistortion generated by said address generating means corresponding todisplay positions of said display means in a first direction amonghorizontal and vertical directions and a plurality of lines connectingthe original pixels in a second direction different from said firstdirection; a step of executing a first interpolation operation at saidintersecting points used as the reference at the time of selection foreach set of the selected original image data; and a step of executing asecond interpolation operation in the horizontal direction with respectto a plurality of interpolated data obtained by the first interpolationoperation and generating new pixel data to be output to the displaymeans based on the correspondences of the addresses obtained by themapping.
 19. An image conversion method including: an address generationstep of generating first interpolation addresses by a first relationshipequation with a coefficient of a coordinate parameter of a firstdirection among horizontal and vertical directions set as “1” andgenerating second interpolation addresses by a second relationshipequation with a coefficient of the coordinate parameter of the seconddirection different from said first direction set as “1”; a firstinterpolation step of selecting a plurality of original pixel dataarranged in said second direction of the input original image by usingsaid first interpolation addresses and repeatedly executing a firstinterpolation operation a plurality of times; and a second interpolationstep of selecting a plurality of interpolated data arranged in saidfirst direction generated by said first interpolation operation by usingsaid second interpolation addresses and executing a second interpolationoperation at the interpolation points to generate new pixel data.